The Effect of Economy Type on Reinforcer Value

Abstract

This article reviews studies investigating the effect of economy type on reinforcer value. In a closed economy, consumption of the reinforcer depends entirely on the subject’s behavior, whereas in an open economy it does not, due, for example, to the provision of free reinforcers after the session. In theory, reinforcers should have higher value in a closed economy than in an open economy. Experimental results relevant to this prediction and methods used to test economy type effects are summarized and discussed here. Studies have tested the effect of economy type on the value of a variety of reinforcers, including food, water, saccharin, various drugs, and video games. Subjects used have varied also and include humans, monkeys, rats, and mice. Whether economy type had an effect on reinforcer value appears to depend on the particular reinforcer studied and on the species used. In general, where there was a difference in reinforcer value across economies, the effect was consistent with the prediction that value should be lower in the open economy. In some studies, however, satiation across economy types may have been responsible for the difference, or at least contributed to it. Potential explanations for the economy type effect, including substitution of future reinforcers for current reinforcers, contingency degradation, anticipatory contrast, and optimal foraging, are discussed.

1. Introduction

Suppose that a person on a limited income must decide between spending money on prescription medication or food. The only way to obtain the medication is by paying for it at the pharmacy. Food can be obtained by paying for it at the supermarket, but for this individual there are several other sources of food, including a supportive circle of family and friends, a government assistance program in which the person is already enrolled, and a local church that the person attends. For this individual, medication is available in a closed economy and food is available in an open economy. In this situation, it seems likely that the individual would choose to spend their money on the medication rather than food. However, if the food economy were closed – i.e., buying it at the supermarket was the only way to obtain it – it might be more difficult to predict how the person would spend their money.

Since the beginnings of behavioral economics, economy type has been recognized as an important determinant of behavior (Hursh, 1978, 1980). Hursh (1980, 1984) has defined an open economy as one in which there is some degree of independence between a subject’s behavior during the session and its total daily consumption of a reinforcer, whereas in a closed economy there is no independence. For example, if the subject can earn food in the operant chamber by pressing a lever during the experimental session and also receives experimenter-provided food after the session, the food economy is open. In contrast, if the subject’s only source of food is that earned during the session, the food economy is closed. As described below, in addition to experimenter-provided post-session reinforcer access, there are other ways of establishing independence between a subject’s behavior and its consumption of the reinforcer.

This primary purpose of this review is to summarize and discuss studies that have investigated the effect of economy type on reinforcer value. Economy type studies not focused on reinforcer value, but on phenomena like schedule effects or reinforcer magnitude effects (e.g., Bullock and Hackenberg, 2006; Collier et al., 1992; Elliffe et al., 1999; Foster et al., 1997; Hall and Lattal, 1990; La Fiette and Fantino, 1988; Timberlake and Peden, 1987; Zeiler, 1999), will not be covered here in detail (see Posadas-Sanchez and Killeen, 2005, for a review of such studies). The current review will focus on reinforcer value because the behavioral economic prediction about the effect of economy type is fundamentally that a reinforcer should have lower value in an open economy than in a closed economy. This is because, in an open economy, reinforcers obtained outside of the schedule of reinforcement in effect during the session, in theory, substitute for those that could be earned by working on that schedule of reinforcement, whereas there are no such substitutes in a closed economy (Hursh, 1980, 1984). A human example of this kind of substitution effect is the person who ordinarily buys a sandwich for lunch, but does not buy one today because there will be free food at an afternoon reception. The later free food substitutes for the sandwich that can be bought now. As will be discussed later, some have questioned whether such substitution can explain all instances of economy type effects, but this view will be the starting framework for what follows. The review will begin with a discussion of how reinforcer value will be defined here, proceed to a description of methods used to manipulate economy type, then to a summary of findings with different kinds of reinforcers, and end with a discussion of possible explanations of the effect.

2. Reinforcer value

As Rachlin (1992) noted, the concept of reinforcer value is implicitly circular unless it is operationally defined. Hursh and Silberberg (2008) reviewed the various ways (e.g., response rate) that this has been attempted in the history of psychology, noting the shortcomings of previous approaches. They proposed defining reinforcer value in terms of elasticity of demand, which describes how quickly consumption of a reinforcer decreases with increases in price. Hursh and Silberberg developed a mathematical model of demand that isolates elasticity from other variables (e.g., reinforcer magnitude) that can affect behavior. They argue that elasticity of demand, isolated this way, reflects the essential value of a reinforcer. Put simply, subjects work harder to defend consumption of high essential value reinforcers (i.e., those with relatively inelastic demand) than they do to defend consumption of lower essential value reinforcers (i.e., those with relatively elastic demand).

Breakpoint on a progressive ratio (PR) schedule (Hodos, 1961) is also thought to reflect the effort a subject is willing to expend for a reinforcer and has therefore also been used to quantify reinforcer value. Indeed, use of demand elasticity or PR breakpoint often yields similar conclusions. For example, Freeman et al. (2014) found that the same manipulation that reduced the essential value of cocaine in monkeys also reduced PR breakpoints for cocaine. Similarly, when Gannon et al. (2018) compared various drug reinforcers in rats, they found a similar rank ordering of drugs whether essential value or breakpoint was the measure of reinforcing effectiveness (i.e., value). Murphy et al. (2010) found in humans that individual differences in elasticity of demand for nicotine predicted individual differences in breakpoints. The main advantage of essential value over PR breakpoint is that the latter is reinforcer magnitude dependent (e.g., Roberts et al., 2002), whereas the former is not (Hursh and Silberberg, 2008).

Preference, as revealed in a choice situation, is the third way that reinforcer value will be operationally defined here. It is assumed that subjects prefer higher value reinforcers over lower value reinforcers (Rachlin, 1992). A strength of preference as a measure of reinforcer value is that it is response-rate independent (Negus and Banks, 2018). There is evidence showing that preference, as a measure of reinforcer value, converges with both demand elasticity and PR breakpoint. For example, individual differences in PR breakpoints or elasticity of demand for drug reinforcers predicts individual differences in drug preference (James et al., 2018; Kearns et al., 2017; Perry et al., 2013; Schwartz et al., 20017). Further, manipulations that decrease PR breakpoints or the essential value of a reinforcer also decrease preference for that reinforcer (Freeman et al., 2014; Kim et al., 2018; Negus, 2005).

The studies reviewed below used one or more of these measures of reinforcer value. Evidence that they are often correlated with each other and that manipulations that affect one measure in a particular way also often affect another measure in the same way suggest that they reflect some common feature of a reinforcer. [See Bickel et al. (2000) for in-depth discussion of the interrelationships among elasticity, PR breakpoints, and preference and how these interrelationships can be understood through demand curve analysis.] This is not to say that these measures always correlate [e.g., see Madden et al. (2007a, 2007b) for instances where they do not] or that there are not other potentially valid measures of reinforcer value (e.g., O_max). But reinforcer value will be operationally defined in terms of these three measures here since they are the measures used in the studies that have investigated the effect of economy type on reinforcer value reviewed below.

3. Methods for establishing open vs. closed economies.

3.1. Experimenter-controlled vs. subject-controlled number of reinforcers per session.

In the original study that manipulated economy type as an independent variable, Hursh (1978) allowed monkeys to work for food that was available for responding on two VI schedules. For one of the schedules, the VI value was constant (Food 1) and for the other (Food 2) it varied across sessions. In Exp. 1, sessions terminated after a fixed duration. In Exp. 2, sessions terminated after a fixed number of food pellets were earned. Hursh found that in the latter condition, response rates on the Food 2 schedule increased as the VI schedule became richer (i.e., the VI value shortened), consistent with previous studies on VI schedule effects (e.g., Catania and Reynolds, 1968). But when sessions terminated after a fixed duration, response rates were inversely related to the richness of the VI schedule. Hursh (1978) viewed the critical difference between these conditions as whether total daily food intake was under the subjects’ control (Exp. 1) or under the experimenter’s control (Exp. 2).

Hursh (1980) described the fixed session duration condition (Exp. 1) of the study described above as a closed economy and the fixed number of pellets condition (Exp. 2) as an open economy. What distinguished these two conditions as closed vs. open economies was total daily food intake being dependent on the subject’s behavior vs. total daily food intake being at least partly independent of the subject’s behavior (Hursh, 1980). It should be noted that here independent does not mean that food was given response-independently, as it is on a VT schedule. In both of the conditions used by Hursh (1978) animals had to work for all their food, on the same VI schedules, and no “free” food was given at any time. Rather, independent here means that monkeys in the fixed number of pellets condition (open economy) were guaranteed the same number of pellets per day regardless of how slowly or how quickly they responded, or how they distributed their behavior across the VI schedules. In contrast, the rate at which subjects responded and how they distributed their behavior across the VI schedules determined how much total daily food they obtained in the fixed session duration condition (closed economy). Hursh (1980) has described the openness of an economy as falling on a continuum. If there is a large degree of independence between consumption and behavior, the economy is said to be more open than when there is only a small degree of independence.

3.2. Extra post-session reinforcers.

Another way to manipulate economy type is to provide post-session access to the reinforcer in the open economy while restricting all reinforcer access to the experimental session in the closed economy. When food is the reinforcer, a simple way of creating an open economy is to give subjects supplemental food in their homecages after each experimental session (e.g., Cassidy and Dallery, 2012). Post-session reinforcers can also be provided in the operant chamber. Hursh et al. (1989) performed an experiment that illustrates different ways in which post-session access to the reinforcer can be provided. In a first phase, monkeys, which lived in their test chambers, pressed a push-plate for food pellets on fixed-ratio (FR) schedules that increased over days from FR 10 to FR 372. These “work” sessions lasted 12 h. In the closed economy, the food obtained by working on the FR schedule was the only food available to the monkeys. In the open economy condition, monkeys received additional food pellets after the work session on a non-contingent fixed-time (FT) 3-s schedule. Demand was less elastic in the closed economy than in the open economy. In a second phase, monkeys again responded for food during work sessions on FR schedules that increased over days from FR 10 to FR 372. In the closed economy condition, this was the only food available to the monkeys. In the open economy, the work session was followed by a period where monkeys could respond for food on an FR-1 schedule. Consistent with the results of phase 1, where post-session food was given non-contingently, providing a cheap source of earned food after the work session caused demand for food during the work session to become more elastic.

3.3. Extra within-session reinforcers.

The degree to which total daily consumption of a reinforcer is dependent on the subject’s interaction with the schedule of reinforcement can also be manipulated by providing extra within-session reinforcers. For example, Imam (1993) trained pigeons to keypeck for food on an FR-50 schedule. In one condition, occasional extra food deliveries were presented according to a variable-time (VT) schedule and in another condition extra reinforcers were available according to a VI schedule where the same response that counted towards the FR schedule also counted towards the VI schedule (i.e., a conjoint schedule was used). Regardless of whether the extra food presentations were scheduled according to a VT or VI schedule, the rate of keypecking on the FR-50 schedule was reduced when the extra food occurred once a minute vs. once every eight minutes. As the extra VT or VI food deliveries constituted a greater proportion of total food deliveries, the pigeons were less inclined to work for food. A similar effect was found in the same study when all extra food was given post-session in the more common open economy arrangement. Imam proposed an “independence quotient” to quantify the degree of openness of an economy. The quotient is simply the total amount of unearned food, whether given post-session or during the session, divided by the total amount of food consumed. A value of zero represents a totally closed economy (no unearned food) and a value of 1 (all food unearned) represents the extreme of an open economy.

In summary, open economies have been established in three ways. In the first method, subjects’ total daily consumption of a reinforcer is determined by the experimenter, rather than by the subject, by allowing sessions to last as long as is necessary to obtained the experimenter-determined number of reinforcers. In the second method, subjects are given post-session non-contingent reinforcers or contingent access to the reinforcer on a less demanding schedule (usually FR 1) than is used during the session (Hursh et al., 1989; Kim et al., 2018). In the third method, subjects are given non-contingent reinforcers or contingent reinforcers on a less demanding schedule during the session. It may be noted that in the second and third arrangements described above, the response-contingent reinforcers are entirely dependent on subjects’ behavior, even if on only an FR-1 schedule. This seems inconsistent with the definition of an open economy as being one in which there are at least some unearned reinforcers (Imam, 1993) or one in which there is at least some independence between a subject’s behavior and total consumption (Hursh, 1980). On the other hand, it seems that if a subject has been working for a reinforcer on a large FR (e.g., FR 100) during the session, whether post-session food pellets are provided on an FR-1 schedule or presented non-contingently (e.g., on an FT 3-s schedule) should be a trivial difference.

4. Potential co-variates of economy type

Before getting into the particulars of how economy type can been manipulated, a word of caution is needed about potential co-variates of economy type that could complicate interpretations of results. For example, in some studies of economy type, many of which focused on schedule effects [e.g., whether response rate increases or decreases with richness of a variable-interval (VI) schedule], procedures were arranged such that animals could obtain more of the reinforcer in the closed economy than in the open economy (e.g., Bullock and Hackenberg, 2006; Collier et al., 1992; La Fiette and Fantino, 1988; Zeiler, 1999). This seems paradoxical because it might be expected that, if anything, subjects should obtain more of a commodity when there are multiple sources of it (e.g., food earned during the session plus free post-session food) than when there is only one. In part, this arrangement was used because food was used as the reinforcer. To allow subjects to obtain enough food to maintain healthy weights, very long (e.g., 23.5 h) sessions were used in the closed economy. Such long sessions often resulted in subjects earning enough food per day to maintain bodyweights at or near ad-libitum levels (La Fiette and Fantino, 1988; Collier et al., 1992; Zeiler, 1999). In contrast, in the open economy conditions, relatively short (e.g., 1 h) sessions were used and only enough supplemental food was provided to maintain weights at 80% of free-feeding.

In studies using such procedures, economy type co-varied with session length and, perhaps more importantly, food deprivation or satiation level. Killeen (1995; Posadas-Sanchez and Killeen, 2005) has argued that satiation, rather than economy type, may be the critical variable controlling behavior in the kind of experiments described above. In closed economies where the reinforcer is food, where sessions are very long, and where the reinforcement schedule is not too lean, subjects start each session at a low deprivation level and reach satiation relatively quickly. In contrast, in open food economies where sessions are short and subjects are maintained at 80% of free-feeding weights, food deprivation level is high at the start of the session and animals may never become sated during the session. It is difficult to discern the effect of economy type per se in studies using these arrangements. Many of these studies that allowed animals to sate in the closed economy, but not in in the open economy, did not focus on reinforcer value and will not be discussed below. However, they illustrate the kinds of potential alternative explanations of results that must be considered when levels of the economy type independent variable covary with another variable.

5. Economy type and the value of different reinforcers

5.1. Economy type and food value

The Hursh et al. (1989) study described above was one of the first to investigate how economy type affects the reinforcing value of food. In addition to finding that post-session access to food made demand for it more elastic, Hursh et al. also explored whether the timing of the extra food was important. In one of the phases of the study, monkeys worked for food in four separate hour-long sessions that occurred over the course of the day. In the closed economy, these work sessions were monkeys’ only opportunity to obtain food. In one of the open economy conditions, following the last work session there was a 20-min period during which food was available on an FR-1 schedule. In another open economy condition, there was a 5-min FR-1 period at the end of each of the four work sessions. Demand for food was more elastic in each of the open economies relative to the closed economy and most elastic in the open economy where there was an FR-1 period at the end of each of the four work sessions. That is, extra food had a bigger impact on demand when it occurred closer in time to when monkeys were working for it. This suggests that extra reinforcers more readily substitute for earned reinforcers when they come sooner rather than later.

Hursh (1991) noted that the monkeys in the open economy combined earned and extra food and ended up “overeating” relative to what they ate in the closed economy. Even when the price of food was relatively low (FR 10) during the work session, and therefore food consumption was nearly unconstrained in the closed economy, the monkeys ate more total food in the open economy. Hursh (1991) wrote that monkeys in the open economy became “quite fat, judging by their bodyweights.” This suggests that satiation could have been responsible for the difference in demand elasticity across economy types. It should be noted that, here, more satiation occurred in the open economy than in the closed economy, which is opposite to the situation in the studies reviewed by Posadas-Sanchez and Killeen (2005), where animals were typically food deprived (e.g., maintained at 80% of free-feeding weights) in the open economies while they were closer to ad lib weight (i.e., non-deprived) in the closed economies. Less deprivation in the closed than in the open economy can occur when closed economy sessions are much longer than open economy sessions (e.g., 23 h vs. 1 h) and animals can acquire food relatively easily. If, however, sessions lengths are equivalent across economy types, the additional food provided to subjects in the open economy may cause them to be nearer to satiation than subjects in the closed economy, as appears to have occurred in some studies using food as the reinforcer (described below).

Carroll et al. (2000) investigated the effects of economy type on PR breakpoint for food in monkeys using a design similar to that of Hursh et al. (1989). In the open economy, supplemental food was placed in the food hopper 30 min after the session wherein monkeys responded for food on the PR schedule. Breakpoints were lower in the open economy than in the closed economy. Carroll et al. wrote that “weights increased slightly” in the open economy, which again raises the possibility that reduced breakpoints were due to less severe food-deprivation in the open economy than in the closed economy.

Nader and Woolverton (1992) allowed monkeys to make mutually exclusive choices between cocaine and food in either an open or closed economy. Monkeys obtained banana-flavored pellets during the choice session. In the open economy, they also received supplemental monkey chow after the choice session, whereas in the closed economy they did not. Food choice decreased (and cocaine choice increased) when supplemental post-session food was given. This is consistent with the notion that opening the food economy decreased the value of food. For two out of the three monkeys, however, increased food choice coincided with decreased bodyweight in the closed economy. For the other monkey, bodyweight increased in the closed economy. The authors report that bodyweight was not statistically associated with choice, but with only three subjects, such a null result is difficult to interpret. In summary, the results of three studies (Carroll et al., 2000; Hursh et al., 1989; Nader and Woolverton, 1992) in monkeys are consistent with the idea that food value is lower in open than in closed economies. However, in each study, variations in food deprivation across economies may have influenced results.

Cohen et al. (1990; Exp. 1) used a yoking method to equate bodyweights across open and closed economies wherein rats worked for food on an ascending series of FR schedules. One rat of each pair was assigned to the closed economy and the other to the open economy. Closed economy sessions lasted 4–6 hours so that rats could obtain enough food pellets to maintain a healthy bodyweight. Open economy sessions lasted 45 min. The amount of post-session food given each day to the open economy partner of the pair was equal to the total amount of food earned by the closed economy partner during its session minus what the open economy partner earned during its session. In this way, the bodyweights of rats were equated across economy types, at least at the start of each session. Though rats in the closed economy tended to respond at higher rates than those in the open economy, there was no significant difference in elasticity of demand for food. The authors of the study noted that this lack of a significant difference in demand elasticity was due to substantial error variance.

Cassidy and Dallery (2012) compared elasticity of demand for sucrose pellets in open and closed economies in rats. They found little difference across economy types. However, satiation and factors that co-varied across economy types complicate the interpretation of results. Subjects were trained to lever press for sucrose pellets on a series of FRs that increased over sessions. In the open economy, sessions lasted 130 min and occurred every other day. The amount of extra-session food was adjusted to keep rats’ bodyweights at 85% of free feeding at the start of each session. In the closed economy, sessions lasted 23 h and were conducted daily. Rats in the closed economy began the ascending FR series at 85% of free-feeding, but once started, no effort was made to control bodyweight. At many of the FRs, rats were able to obtain several hundred food pellets per session. For example, at FR 1, rats in the closed economy earned 800–1200 pellets per session. At the start of the next session on the following day, these rats were likely at or above their free-feeding weights. In contrast, rats in the open economy were given a day off so that they would return to 85% of free-feeding weight before their next session. Rats in the open economy also often earned several hundred pellets per session, at least at the lower FRs. As Cassidy and Dallery note, “satiation probably occurred in both the open and closed economies, and this may account for the similarity of the curves obtained during both economy types.” In summary, studies of the effect of economy type on food reinforcer value in rats have yielded ambiguous results.

In a more recent study with mice, Soto et al. (2016) investigated the effect of economy type on elasticity of demand for food in mice. They equated food-deprivation level at the start of each session across economies by only running sessions on days when subjects were at 85% of free-feeding weights. If a mouse was not at 85% of its free-feeding weight on a particular day, no session occurred and the subject was given an amount of food that would bring its weight to 85% of free-feeding the next day. While subjects in each economy type weighed the same at the start of the session, it is likely that motivation varied between the groups during the session. In the closed economy, subjects could earn several hundred pellets over the course of an 11-h session, whereas in the open economy the session ended after either 20 pellets were earned or 30 min elapsed. If anything, this difference should have made satiation more likely during closed economy sessions than during open economy sessions. Despite this, rats worked harder as price increased in the closed economy than in the open economy. In this case, it seems that satiation can be safely ruled out as a potential explanation for reduced food essential value in the open economy. Session length (11 h vs. 30 min or shorter), however, differed across economy types.

5.2. Economy type and water value

Ladewig et al. (2002) performed an experiment intended to compare the elasticity of demand for water across open and closed economies in water-deprived rats. Subjects lever pressed for water on a series of FRs that increased from 10 to 100 over sessions, which lasted 2 h each. Economy type was manipulated within subjects over phases. In the closed economy, rats’ only source of water was that earned during the session. In the open economy, rats were given 10 or 30 min access to post-session supplemental water in the homecage. In different open economy conditions, the supplemental water was presented either immediately after the work session or after delays of 2 h, 18 h, or 21 h. Rats’ demand for water was less elastic in the closed economy than in any of the open economy conditions. However, it is likely that this result was due to differences in water-deprivation level at the start of the session rather than to an economy type effect specifically. The authors unfortunately did not measure the amount of water that rats drank in the homecage, making it impossible to determine overall water intake levels across economy types. But they found that demand for water became more elastic as the delay from the work session to the presentation of the supplemental homecage water increased. This is counterintuitive from a behavioral economic perspective because long delayed extra water should have less readily substituted for work session water. As Ladewig et al. noted, the longer the supplemental water was delayed after one work session, the closer it was to the start of the next work session. This suggests that the outcome observed was akin to a pre-feeding (or, pre-watering here) effect – i.e., differences in work session behavior were due to differences in current water-deprivation level.

As the studies reviewed above show, food and water are difficult reinforcers to work with when studying the effect of economy type. Deprivation level co-varied with economy type in about half of these studies. It is hard to estimate the extent to which deprivation level or economy type contributed to effects observed in such studies. It is worth reiterating, however, that a clear effect of economy type was observed when differences in food deprivation level could be ruled out as a potential alternative explanation (Soto et al., 2016). Equating deprivation and satiation across economies is a major challenge. For example, to equate rates of within-session food satiation, subjects in both economies must start each session at the same food-deprivation level, the same contingencies must be operative during the session, and sessions must be of the same length. This is difficult to achieve when subjects in the open economy have an additional source of food that subjects in the closed economy do not have and the energy from that extra food can be stored in the body (e.g., in fat stores) over relatively long periods. Attempts to equate the total amount of food given per day across economy types necessarily introduces another variable, such as session length or session frequency, that covaries with levels of the economy type independent variable. This conundrum might be resolved by using a nonnutritive reinforcer that is not stored in the body over long periods, such as saccharin, or a drug reinforcer with a short half-life, to study the effects of economy type. Studies that used these kinds of reinforcers are reviewed below.

5.3. Economy type and saccharin value

Kim et al. (2018) compared elasticity of demand for saccharin in open vs. closed economies. During a 3-h work session, rats lever pressed for saccharin on a series of FRs that gradually increased over two-session blocks from 1 to 96. For rats in the open saccharin economy, this work session was followed shortly by a second session each day that occurred in a different operant chamber where rats could press a lever for saccharin on an FR-1 schedule. Rats in the closed economy were also placed in another chamber for a second session, but they had no access to saccharin there. The essential value of saccharin was significantly lower in the open economy than in the closed economy. Gunawan et al. (under review) have recently replicated this finding with saccharin using similar procedures.

Overall, the rats in the open economy consumed more total saccharin than the rats in the closed economy. Bodyweights did not differ across economy types, which is not surprising because saccharin has no calories. But might the saccharin consumed on one day have made rats partly saccharin-sated at the start of the work session the next day and therefore less inclined to work for it? Saccharin is not stored in the body in the way that energy from food is stored in the body. The half-life of saccharin in rats is only about 30 min (Sweatman and Renwick, 1980; Renwick, 1985), which would mean that any saccharin consumed on one day would have been eliminated well before the start of the next day’s work session. Additionally, earlier research indicates that saccharin satiety in rats is controlled by immediate oral sensory stimulation rather than by longer-term post-ingestive consequences such as fullness of the stomach or overhydration (Mook et al., 1981). The short-half life and the lack of post-ingestive consequences for saccharin drinking suggest that the effect of economy type on saccharin reinforcer value observed by Kim et al. (2018) was not due to differences in satiation across economies.

5.4. Economy type and drug reinforcer value

In some of the clearest examples of the effect of economy type on reinforcer value, Mitchell and colleagues (1994, 1995, 1998) performed a series of experiments using human subjects and cigarettes or coffee as the reinforcer. In the 1994 study, participants mouse-clicked on a concurrent random ratio (RR) RR schedule for points that could be converted into cigarettes or money. The RR for cigarettes varied from 1.3 to 16, whereas the RR for money was always 4. In the closed economy, only the cigarettes earned on the RR schedule could be smoked during a 6-h post-session period during which subjects remained in the lab. In the open economy conditions, subjects were informed prior to the work session that they would be given either 5 or 10 free cigarettes which they could smoke during the 6-h post-session period. Demand for cigarettes was more elastic in the open cigarette economies than in the closed economy. Mitchell et al. (1998) replicated this result with cigarettes in a later study. They also found, using a similar design, that demand for coffee was more elastic in an open coffee economy than in a closed economy (Mitchell et al., 1995).

Satiation did not likely play a role in the Mitchell et al. studies. A within-subjects design was used where each subject experienced all levels of economy type variable. At least two days separated test sessions, ruling out the possibility of between-session satiation differences across economy type conditions. The extra nicotine or caffeine obtained during open economy sessions would have been eliminated well before the start of the next session. Further, the earned and free reinforcers were given post-session, ruling out the possibility of different within-session changes in satiation across economy types. It seems safe to conclude here that it was the expectation of future free cigarettes or coffee that made demand for these reinforcers more elastic.

Greenwald and Steinmiller (2009) used a concurrent drug vs. money choice procedure similar to that used in the Mitchell et al. studies but with hydromorphone (Dilaudid) as the drug reinforcer rather than cigarettes or coffee. They found that when the non-drug alternative was $2, informing subjects of the post-session availability of hydromorphone made demand for hydromorphone more elastic. As in the Mitchell et al. studies, it seems safe to conclude that it was the expectation, produced by the experimenter’s instructions, of free post-session drug that was responsible for the effect of economy type here.

In contrast to the significant effect of economy type on cigarette, coffee, and hydromorphone value in humans, most studies in animals have not found reduced drug reinforcer value in an open vs. closed economy. For example, in the same experiment by Kim et al. (2018) described above, another group of rats worked for intravenous cocaine infusions on a series of FRs that increased over days from 1 to 96. In the closed economy, this work session was rats’ only opportunity to obtain cocaine. In the open economy, the work session was followed by an additional 3-h period in a different chamber where rats could self-administer cocaine on an FR-1 schedule. There was no difference in demand for cocaine across economy types. Gunawan et al. (under review) also did not find a difference in demand for intravenous heroin across open vs. closed economies using similar procedures.

Banks and Negus (2010) performed an experiment that investigated monkeys’ preference for cocaine over food across conditions that could be described as closed vs. open cocaine economies. During a two-hour choice session, monkeys worked on a concurrent FR FR schedule where food was available on an FR-100 schedule and cocaine infusions were available on an FR-10 schedule. During the baseline condition, the choice session was monkeys’ only opportunity to obtain cocaine. This baseline condition fits the definition of a closed cocaine economy. On alternate weeks, the two-hour choice session was followed by a 21-h supplemental session where monkeys could self-administer cocaine on an FR-10 schedule. The maximum number of infusions available per supplemental session was manipulated over weeks by varying the length of the post-infusion timeout period from 7.5 to 30 min. This situation could be described as an open cocaine economy because subjects could obtain cocaine during the choice session and also during the 21-h supplemental session. Each seven-day period with supplemental cocaine sessions (open economy) was followed by a week of baseline conditions (closed economy). Consistent with the results of Kim et al. (2018) described above, supplemental cocaine had no effect on the degree of preference for cocaine over food. When the shortest post-infusion timeout period was in effect, supplemental cocaine acted to reduce the total number of choices made during the choice session, but did not change the relative allocation of responses across alternatives.

In the study described earlier where Carroll et al. (2000) found that opening the economy for food reduced monkeys’ PR breakpoints for food, they also tested the effect of economy type on responding for two orally self-administered drug reinforcers, PCP and ethanol, using similar procedures. Post-session access to PCP had no effect on breakpoints for PCP. Breakpoints for ethanol, however, were lower in an open vs. closed economy. It is perhaps worth noting that ethanol has calories and that the monkeys in the Carroll et al. study were food deprived at the start of the experiment. It is possible that ethanol in this situation was acting partly like a nutritive reinforcer here, whereas PCP, which has no nutritive value, was a more pure drug reinforcer.

In summary, with drug reinforcers, the effects of economy type appear to depend on the particular drug and species being studied. In humans, opening the cigarette, coffee, or hydromorphone economy reduced the value of these reinforcers. In animals, the effect of economy type has been tested with cocaine, heroin, PCP, and ethanol. An effect of economy type was only observed for ethanol. Possible reasons for these differences across reinforcers and species are discussed below.

5.5. Economy type and sensory/conditioned reinforcement.

Roane et al. (2005) investigated the effect of economy type in two developmentally disabled individuals. For one subject, the reinforcer was the opportunity to play a videogame and for the other it was the opportunity to watch a cartoon video. The operant was completing math worksheets for one participant and performing an envelope-sorting task for the other. A PR schedule was used during the work session. A multiple reversal design was used where each individual was exposed to the open and closed economies twice each in phases lasting several sessions. In the open economy, participants were informed prior to the PR session that if they wanted to play the game or watch the video, they would have to work for it. In addition, they were told they would have the opportunity to watch or play for a fixed number of minutes after the session regardless of how much work they did. In the closed economy, they were only told that if they wanted to play or watch, they would have to do the work. A clear and consistent effect of economy type was observed, with both participants working harder on the PR schedule in the closed economy than in the open economy. Like the Mitchell et al. studies, this is another experiment reporting a clear effect of economy type in humans.

6. Potential explanations for the effect of economy type on reinforcer value

Section 3 described the operations that have been used to establish open vs. closed economies. The essential feature distinguishing open from closed economies was the degree of dependence of a subject’s reinforcer consumption upon the subject’s behavior. This distinction describes aspects of the environment, but is silent with respect to the underlying mechanisms that might cause the organism to behave differently in an open vs. a closed economy. This section will consider potential behavioral mechanisms that could explain why subjects may, or may not, value reinforcers differently in open vs. closed economies.

6.1. Substitution

The behavioral economic account of the effect of economy type is based on substitution (Hursh, 1984). In an open economy, a future, cheaper source of food substitutes for the current, more expensive source of food. In a closed economy, there is no such substitute. This explanation requires that subjects are capable of learning and comparing the prices of the reinforcer available during vs. after the session. It also requires that an event that is delayed (post-session access to the reinforcer), by minutes or even hours, influences current behavior. For humans, this account does not seem problematic. Indeed, it is not hard to imagine a person in the Mitchell et al. studies thinking “why work for cigarettes now and forgo money when I’ll get free cigarettes after the session?” Perhaps the ability to reason in this way is what accounts for the clear and consistent effects of economy type in studies using human subjects (Mitchell, et al., 1994, 1995, 1998; Roane et al., 2005).

For animals, however, some have questioned whether the substitution account is plausible. For example, Posadas-Sanchez and Killeen (2005) wrote that an “explanation in terms of deferred consumption assumes foresight, planning, and some measure of self-control on the part of animals that fail to show even modest self-control over delays of dozens of seconds.” Timberlake (1984; Timberlake et al., 1987) showed that availability of future free or cheap food only influenced rats’ barpressing for food on a PR schedule if that future food was delayed by less than 32 minutes (see also Bacotti, 1976). These results suggest that if any integration of current and future reinforcers occurs, as the substitution view requires, it must occur over relatively short (< 30 min) intervals in rats. On the other hand, there is evidence from other studies showing that rats can integrate reinforcer costs, and adjust their behavior in anticipation of changing costs, over much longer periods, including periods as long (Johnson and Collier, 1994) or longer than 24 h (Morato et al., 1995; for review, see Collier and Johnson, 1997). Procedural differences likely account for these seemingly discrepant results. We do not yet know why animals seem able to intergrate reinforcers over long periods in some situations, but fail to do so in others. As Timberlake et al. (1987) concluded, there is probably no single time horizon that defines the limit over which animals can integrate current and future rewards. Species, reinforcer type, cost, behavior system (e.g., feeding vs. drinking), and a number of other variables likely determine the time horizon.

6.2. Contrast

An alternative explanation of economy type effects on reinforcer value is that anticipatory contrast occurs in the open economy. An example of anticipatory contrast comes from the observation that when access to a saccharin solution is followed by access to a preferred sucrose solution, rats drink less of the saccharin solution than when it is not followed by something better (for review of contrast effects, see Flaherty, 1982). Anticipatory contrast can also be seen when the opportunity to respond on a relatively lean reinforcement schedule is followed by access to a richer reinforcement schedule (for review, see Williams, 2002). For example, Williams (1992; Exp. 1) trained rats to respond on a VI 1-min schedule for food in the presence of a discriminative cue during a component that lasted 10 min. Response rates in this VI component were lower when it was followed by a second 10-min component (signaled by a different cue) during which food was available on a richer FR-1 schedule than when it was followed immediately by a second 10-min component wherein extinction was in effect.

The design of Williams’s (2002) experiment closely parallels those of some of the economy type experiments reviewed above (e.g., Kim et al., 2018; Carroll et al., 2000; Hursh et al., 1989) except with regard to the length of components and the time between components. In the economy type experiments, the analogues to the components of the multiple schedule used by Williams were much longer than 10 min and there was sometimes a lag between the lean schedule and the rich schedule. However, the key feature of the manipulation that produces the effect – cheap reinforcers follow expensive reinforcers – is common to both kinds of experiments.

If the economy type effect is a form of anticipatory contrast, what explains anticipatory contrast? Potential mechanisms underlying contrast effects have been the topic of lengthy previous reviews (e.g., Flaherty, 1982; Williams, 1983, 2002). Various possibilities, including reinforcer habituation, frustration, and matching, have been considered and discarded (Williams, 2002). There does not appear to be a single unambiguous explanation for the phenomenon, and therefore it is difficult to say whether the same mechanism underlies anticipatory contrast and economy type effects. This seems like a potentially profitable area of future research.

6.3. Contingency

A third possible explanation for lower reinforcer value in open vs. closed economies is contingency degradation. Rescorla (1968) demonstrated the importance of contingency in Pavlovian conditioning. The strength of responding to a conditioned stimulus (CS) paired with an unconditioned stimulus (US) was inversely correlated with the probability of the US occurring in the absence of the CS. US presentations outside of the CS weakened the correlation, or contingency, between the CS and US. This framework has also been applied to operant conditioning, where response-independent reinforcers have been shown to reduce instrumental performance (Colwill and Rescorla, 1986; Dickinson and Charnock, 1985; Hammond, 1980; Imam, 1993). In a closed economy, there is a high (even perfect) correlation between the operant response and the consequent reinforcer. In an open economy, delivery of free reinforcers reduces the size of this correlation. Put another way, closed and open economies differ with regard to the feedback relation between consequences and behavior (Hursh, 1984).

Knowing what explains contingency degradation could help to determine whether economy type effects are a kind of contingency effect. One potential explanation of the contingency degradation effect is that the response-independent reinforcers become associated with the context and this causes the context to elicit competing behaviors that interfere with the operant (Dickinson and Charnock, 1985). This competing behavior account might possibly explain results of economy type studies where the earned and extra reinforcers were delivered in the same context (e.g., Carroll et al., 2000; Hursh et al., 1989). But it cannot easily account for the results of experiments where the work session occurred in an operant chamber and the post-session reinforcers were obtained in either the homecage (e.g., Soto et al., 2016) or a second, discriminably different operant chamber (e.g., Kim et al., 2018). Furthermore, experiments concerned with the contingency effect also indicate that context-conditioned competing behaviors cannot explain all instances of the effect (Colwill and Rescorla, 1986; Dickinson and Mulatero, 1989). Additional studies (Colwill, 2001; Rescorla, 1992) have considered, and ruled out, alternative conditioning-based explanations for the contingency degradation effect, but no unambiguous account has emerged. Because it is unclear why contingency effects occur, it is difficult to say whether economy type effects can be reduced to contingency effects.

Williams (2002) concluded his review on contrast with the observation that anticipatory contrast and contingency degradation effects are similar in that both result from the presentation of additional reinforcers. He writes, “The major difference is that contingency experiments present the additional reinforcers in the presence of the same conditioned context that is present for paired reinforcers, whereas contrast experiments present the additional reinforcers in temporally adjacent periods cued by different stimuli. But the difference is surely a matter of degree….” This reasoning could easily be extended to economy type experiments, where the effect is also produced by additional reinforcers. The three kinds of experiments differ in terms of when and where those additional reinforcers occur. That the source of the effect – extra reinforcers – may be the same in all three phenomena may suggest that a similar process could underlie all three. Identifying that common process could be a challenge, however, because as described above, the mechanism underlying contrast or contingency effects considered separately has not yet been established.

6.4. Patch exploiting

Perhaps a broader view, one less focused on mechanisms and more on function, could help in understanding economy type effects in animals. Collier and Johnson (1990, 1997), drawing on foraging theory, viewed animals in operant experiments as benefits maximizers and patch exploiters. In one especially relevant study (Johnson and Collier, 1989), they investigated rats’ foraging strategies when two “patches” of food were sequentially available. Rats first pressed a “search” bar on an FR-5 schedule. Completion of the search requirement produced one of two patches, randomly selected. The cue lights above the active patch illuminated and rats could press the bar associated with that patch. The FR schedules in effect in each patch varied over phases. Once a patch was presented to a rat, it could either accept the patch and bar press for food, or reject the patch and wait (at least 10 min) for a better patch. Johnson and Collier found that the difference in “profitability” (in terms of grams per press) between patches predicted both the likelihood that a subject would accept a patch and the rate at which the subjected pressed the bar in that patch. For example, when both Patch 1 and Patch 2 were associated with an FR-40 schedule, rats accepted Patch 1 on 90% of occasions and pressed the Patch 1 bar at over 70 responses per minute. In contrast, when the same FR 40 operated in Patch 1, but an FR 10 operated in Patch 2, rats only accepted 25–30% of Patch 1 opportunities and they bar pressed in Patch 1 at only about 40 responses per minute. Johnson and Collier identified the relative profitability of patches as the controlling variable. It may be noticed that the design and outcome of this experiment is similar to that of Williams’s (1992) anticipatory contrast experiment. Rather than accounting for their results in terms of contrast, however, Johnson and Collier concluded, on the basis of this and several related findings, that foraging animals make cost/benefit analyses and optimize their foraging over fairly large time windows.

Johnson and Collier’s framework could be readily adapted to the economy type experiments in animals reviewed above. For example, in the Kim et al. (2018) open saccharin economy, the work session (where rats could press a lever for saccharin on an FR that was as high as 96) was analogous to Patch 1. The post-work-session period, which occurred in a different chamber and where the saccharin FR was always 1, was analogous to Patch 2. According to Johnson and Collier’s analysis, the difference in profitability between these patches should have made rats in the open economy more likely to reject Patch 1 opportunities (the work session) and to respond more slowly there than rats in the closed economy who only had one patch to exploit. Indeed, open economy rats could have maximized the benefit/cost ratio by not responding at all in the work session and confining all their saccharin consumption to the second, cheap saccharin session. They did, however, respond during the work session, suggesting they behaved suboptimally. Rats in the Johnson and Collier study also often accepted the less profitable patch when the optimal strategy was to always reject it. They noted such suboptimal choice might be expected when the costs for suboptimal behavior are low. Viewing animals as efficient patch exploiters (see Collier and Johnson, 1997, for review on this topic) and viewing the different constraints on access to the reinforcer across open and closed economies as differences in access to patches of varying profitability could provide a useful framework for interpreting the results of other economy type studies in animals.

6.5. Possible reasons for lack of effect of economy type

Why might an effect of economy type be observed in animals for some reinforcers, but not others, under the same conditions? For example, in the same study, Carroll et al. (2000) found the expected effect of economy type on responding for food and ethanol, but not on responding for PCP. Similarly, Kim et al. (2018) and Gunawan et al. (under review) found the expected effect when saccharin was the reinforcer, but not when cocaine or heroin was the reinforcer. One commonality among the failures to observe an effect of economy type is that the reinforcer was a drug of abuse with no nutritional value.

This suggests that the timeframes over which animals can integrate reinforcers are longer for food and saccharin than they are for drugs. That there may be different time windows for different reinforcers is consistent with Timberlake et al.’s (1987) suggestion that animals’ time horizons vary over behavior systems. Perhaps animals’ feeding and drinking systems have evolved in a way that makes integration over relatively long timeframes possible. This seems to make intuitive sense because with food, for example, an energy deficit now can be made up for later and the “books are balanced” over the course of hours or even days. But take, for example, a reinforcer like warmth in cold rats (Weiss and Laties, 1960). Would access to extra heat after the session balance out the “heat deficit” experienced now? Or, for a more extreme example, take the case where lever pressing is negatively reinforced by shock avoidance. Would the availability of cheap shock-free periods after the session make up for the “safety deficit” experienced now? The answers to these questions are unknown, but it seems that animals might use different “accounting methods” for food vs. warmth or shock avoidance. Perhaps drug reinforcers are more similar warmth or avoidance of shock than they are to food in this regard. The answers to these questions will depend on future research.

7. Conclusion

It has been over 40 years since Hursh’s (1978) original study of the effect of open vs. closed economies on behavior. The distinction between economy types has shown that the broader context of reinforcer constraints or availability in which the subject finds itself can be just as important a determinant of behavior as the more immediate factors (e.g., discriminative stimuli, schedule of reinforcement, etc.) thought to control responding. Hursh (1978) showed that what was thought to be a general principle of behavior – response rates increase with increased schedule richness – was actually specific to one kind of broader context of reinforcer availability. The present article has reviewed subsequent studies that have compared reinforcer value in open vs. closed economies. These studies have shown that for many reinforcers (but not all), value is also a function of the broader context of reinforcer constraints and availability. Without the concept of economy type, it would be hard to make sense of these and other results. This is an example of how the concept of economy type has significantly contributed to the field of learning and behavior.

The distinction between open and closed economies is useful not only for understanding behavior in the lab, but also for understanding the real-world choices a person makes. For example, for an individual making a choice between a drug and non-drug alternative reinforcer, knowing the economy type for each alternative is information that would help predict how the individual will behave. If the drug can only be obtained from one source, whereas the non-drug reinforcer can be easily obtained from more than one source, the individual would be expected to choose the drug often. Conversely, if the drug can be obtained easily from more than one source, but the non-drug alternative can only be obtained from one source, decreased drug choice would be expected. It is only by appreciating the broader contexts – i.e., economies – in which such choices occur that we can fully understand them.

Highlights.

• This article reviews research on the effect of economy type on reinforcer value.

• Reinforcer value is often lower in the open economy than in the closed economy.

• However, differences in satiation may account for the results of some studies.

• Potential explanations of the economy type effect are discussed.