The Relevance of Operant Behavior in Conceptualizing the Psychological Well-Being of Captive Animals

Abstract

The term “psychological well-being” is used in reference to husbandry with animals in human care settings such as research, agriculture, and zoos. This article seeks to clarify and conceptualize the term based upon two approaches that draw from several bodies of literature: the experimental analysis of behavior, experimental psychology, animal welfare and husbandry, farm animal behavior, zoo husbandry, and ethology. One approach focuses on the presence of problem behavior such as stereotypies, depressive-like behavior, and aggression, and emphasizes the conditions under which aberrant behavior in animals under human care occurs. The second approach examines what might be considered wellness by emphasizing opportunities to engage with its environment, or the absence of such opportunities, even if problematic behavior is not exhibited. Here, access to an interactive environment is relatively limited so opportunities for operant (voluntary) behavior could be considered. Designing for operant behavior provides opportunities for variability in both behavior and outcomes. Operant behavior also provides control over the environment, a characteristic that has been a core assumption of well-being. The importance of interactions with one’s environment is especially evident in observations that animals prefer opportunities to work for items necessary for sustenance, such as food, over having them delivered freely. These considerations raise the importance of operant behavior to psychological well-being, especially as benefits to animals under human care.

The issues surrounding the humane management of animals in human care (also called “animal care settings” and “captive animal settings”) are global and have been in the public eye for some time. The use of animals in such settings varies widely and includes agriculture, education, sanctuary, and companionship. Indeed, animal care concerns are relevant also to animal research because the use of animals in highly controlled settings is often critical for achieving valid scientific results.

One controversial facet of animal welfare is the promotion of psychological well-being (PWB). As an example, U.S. federal law, specifically the Animal Welfare Act (AWA), includes protecting the health and welfare of vertebrates used in research, but excludes rodents, birds, and animals used in agriculture. Specific mention of PWB in the Animal Welfare Act, however, is reserved for nonhuman primates and dogs (see e.g., The Guide for the Care and Use of Laboratory Animals (National Research Council, 2011; Silverman, Suckow, & Murthy, 2014) in U.S. legislation but the U.S. National Institutes of Health and under the auspices of the Public Health Service (another federal branch) extends such considerations to a much larger range of species, including rodents.

Within the United States and globally, discussions of PWB for vertebrates including rodents and birds have abounded (e.g., Baumans, 2005; Ferdowsian, 2011; Holden, 2000; Nolen, 2001; Pekow, 2005; Stokstad, Malakoff, & MacNeil, 2000; Turner, Lewis, & King, 2003; Weed & Raber, 2005). Legislative guidelines and accreditation standards for animal welfare and PWB for animals in human care have been adopted by a number of countries and international organizations (see, e.g., European Union’s European Commission, Australia Department of Agriculture, Water, and Environment, World Association of Zoos & Aquaria). We do not intend to review these guidelines. Rather, we wish to address concerns for animal welfare and PWB by using the experimental literature to frame the term “psychological well-being” as a construct that examines voluntary behavior through the lens of reinforcement processes. This approach will identify ways to provide an animal with choices in different aspects of its life.

The expression “psychological well-being” is used colloquially, scientifically, and legally. Because these arenas view the construct differently, its definition has not been clearly explicated. For example, the U.S. Department of Agriculture (USDA) developed, and subsequently amended (1970, 1976, 1985, 1990, 2002, 2007–8, 2013, 2017) the 1966 Animal Welfare Act (AWA) to protect the welfare of animals under human care. The AWA repeatedly refers to care that should be directed to promote an animal’s “health and well-being,” and gives specific recommendations about enhancing well-being in dogs and primates, but this act does not define "well-being" except by example.

Welfare usually refers to maintenance of biological needs, such as temperature regulation, cage materials and size, and an absence of health-related suffering or physical maltreatment in an animal (Silverman et al., 2014). In the Five Freedoms of Animal Welfare (Mellor, 2016; Webster, 1994, 2005), an internationally recognized set of standards originally developed by the British Farm Animal Welfare Advisory Committee (see 2010 for most current form). Welfare refers to freedom from hunger and thirst, discomfort, pain, fear and distress, and the freedom to express normal behavior. In other words, welfare seems to include mostly basic needs, such as those associated with safety and comfort (Kagan, Carter, & Allard, 2015; Mellor, 2016; Whitham & Wielebnowski, 2013). The freedom to express “normal behavior” is part of what we hope to address here, not by attempting to define normal behavior but rather by drawing from the wide range of behavior that animals are capable of acquiring.

The first defining characteristics of poor welfare are frequently self-evident and noncontroversial: an emaciated body, open or infected wounds or tumors, and poor grooming, for example, indicate poor health and occasion immediate veterinary care. Evidence of pain or suffering is less straightforward. Although there are species differences in expression of pain, there are some general behavioral indicators, e.g., reduced activity, withdrawal, reduced eating, postural changes that guard the affected area, vocalizations, and lack of grooming. Physical indicators of pain such as porphyrin staining (red staining around the eyes and nose in rats; National Research Council [NRC], 2011) are also possible. However, an animal can also be in pain and not demonstrate visible signs (NRC, 2011). A general absence of these signs usually indicates that an animal is in good health, but does it mean that an animal is psychologically well?

Pekow (2005) described well-being as the absence of excessive stress. Stressors in animal care settings can be related to housing and husbandry, including constraints to feeding, the presence of certain types of ambient noise, odors, and poorly regulated temperature of the home environment. In addition, handling by animal care staff, experimental procedures, and opportunities for social interactions contribute to wellness or its absence. A number of these factors are species-specific and nuanced, but the question of whether an absence of stress is equivalent to PWB is still unanswered.

The present article represents an attempt to elucidate what is meant by PWB from a behavioral perspective. We attempt to describe more carefully what wellness with animals under human care might be using two approaches. In both, we begin with a focus on behavior, as opposed to inferring a psychological state, where possible. In the first approach, several bodies of literature describing the conditions of captivity that influence the presence of abnormal behavior will be summarized and integrated. This “absence of unwellness” approach will explicate the conditions for three classes of aberrant behavior that are most relevant to PWB: 1) stereotypies, 2) unprovoked aggression, and 3) inactivity. These behavior classes typically do not interfere with the animal’s viability, and as such the concerns that these raise for the animal may be viewed as less important than overtly visible health problems even though these complex symptoms can alert observers to a problem. Thus, it seems that PWB, on one level, might refer to observable signs of possible “unwellness” that occur independent of issues related to physical health.

A second approach considers issues that go beyond problematic behavior by examining the benefits of an interactive environment. It begins with exploring claims that animals benefit from enriched environments and by also determining the extent to which opportunities for operant behavior are necessary. By operant behavior, we mean any behavior that is sensitive to its consequences. It can be as simple as a lever-press or as complex as elaborate foraging or stalking activities, as long as its subsequent likelihood is influenced by the outcome. It closely maps onto what is often referred to as “voluntary” behavior.

To see where such behavior might be at play, consider the laboratory rat in a typical behavioral study. It resides within its cage for a large portion of the day, is maintained at a healthy weight, has access to medical care, participates in an experiment for a 30-min block of the day, and then is returned to its home cage. The rat shows no signs of poor health, stereotypies, or unprovoked aggression, etc., but the question remains whether the rat is psychologically well, whether there are meaningful opportunities to engage with its environment. Some argue that confinement of animals to relatively barren conditions with little opportunity for stimulation makes the animal “unhappy” or depressed. To reduce anthropomorphism, it is necessary that such claims be evaluated and, if scientifically substantiated, acted upon empirically. It is worth noting that although rodents in behavioral experiments represent a small minority of animals used in research (Balcombe, 2006; Ellenbroek & Youn, 2016; Hånell & Marklund, 2014; Nicoll & Russell, 1990), they may also represent the best treated. Their day includes an opportunity to leave the home cage and participate in an alternative activity. Moreover, their caloric intake is restricted, so they do not become obese and suffer the consequent health problems like diabetes and heart disease (Dixit, 1999; Martin, Ji, Maudsley, & Mattson, 2010).

The Problem Behavior Approach

One approach to determining whether an animal is psychologically well is to compare behavior in captivity with behavior observed in the wild. Behavior that appears in captivity, but not in healthy animals in the wild, is considered abnormal and may indicate unwellness (e.g., Baumans, 2005; Broom, 1999; Poling, Nickel, & Alling, 1990; Wemelsfelder, 1999). PWB, in this case, refers to an absence of abnormal behavior.

Three classes of behavior that are considered abnormal include stereotypies (invariant, repetitive behavior that appears to have no function), depressive-like behavior (inactivity, diminished consumption of sweet solutions), and aggression (harmful or injurious behavior toward another organism, self, or object that is not predatory, territorial, or reproductive in nature; Broom & Johnson, 1993; Buchenauer, 1981; Hughes & Duncan, 1988; Novak & Suomi, 1988; Young, 1999). Although illness and disease can influence the presence of these behavior, so can environmental factors (Balcombe, 2006; Broom, 1999; Gross, Richter, Engal, & Würbel, 2012). These behaviors, and the conditions under which they occur, are reviewed here.

Stereotypies

One source of abnormal behavior observed in animals under human care is ethological in nature: restriction of the expression of species-specific behavior may result in invariant, repetitive, purposeless movements. These stereotypies include route tracing, bar biting, jumping, tongue chewing, pattern swimming, sham chewing and other repetitive, invariant seemingly functionless behavior (see Broom, 1998; Garner, 2005; Gross et al., 2012; Mason & Rushen, 2008). There indeed are species differences in stereotypies. For example, the highest frequencies of route tracing are observed in species that travel larger distances in the wild, such as polar bears and lions. Other species that travel less in the wild (e.g., mink and arctic foxes) show less route tracing in captivity (Clubb & Mason, 2003, 2007).

Environmental conditions also increase the likelihood of stereotypies and interact with species-related factors. Some of these environmental factors include reduced cage size, diminished opportunities to interact with the environment, and placing access to reinforcers under predictable, but noncontingent schedules of reinforcement. Each of these will be addressed.

Cage size

Much of the research on stereotypies involves restriction of living quarters, especially cage size or surface area available in the home cage. Indeed, the inverse relation between space and stereotypies is well established (e.g., Bachmann & Stauffacher, 2002; Dantzer, 1986; Kitchen & Martin, 1996; Lawrence & Terlouw, 1993; Marshall et al., 2016). Enclosure size requires consideration of each species’ natural habitat, ecology, and behavior. It is obvious that larger animals will need greater space than smaller ones but the quality of the space can be as important as the size, as discussed below.

Restricted environmental opportunities

Wemelsfelder (1999) reported that early in confinement, as time in a barren environment increases, stereotypies tend to be directed toward objects in the external environment, so cage gnawing, for example, might appear. This problem is especially acute in relatively barren environments. With prolonged confinement, however, the stereotypies become less directed toward the outer environment and more towards the animal’s body, so paw biting may become more prominent. The barrenness of the environment can play another role because if the option of interacting with an external source is not available. In this case, then activities that might have been directed at another animal or object become directed against the animal body. Mason, Clubb, Latham, and Vickery (2007) suggest that when behavior becomes directed toward the organism itself, the topography is often similar to behavior the organism would engage toward objects in the natural world. For example, ocelots engage in fur plucking prey in the natural world. Without prey, they may direct fur plucking toward their own bodies. Because of this observation, some (e.g., Berkson, 1983; Wemelsfeder, 1990; Meagher & Mason, 2012) argue that self-stimulation, as well as stereotypies directed towards the outer environment (Krzak, Gonyou, & Lawrence, 1991) is the animal’s way of “counteracting boredom.” A more objective manner of explaining this, however, is to note that stereotypies may have an operant component, that is, stereotypies may have some reinforcing properties (intrinsic or otherwise). This means the act of performing the stereotypy is itself reinforcing and when the natural object is not available another one will substitute, even if it is the animal’s own body.

Intrinsically reinforcing stereotypies have been observed in children and individuals with developmental disabilities, such as Autism Spectrum Disorder (e.g., Hermeun & Frtth, 1991; Kanner, 1943; Kohls, Yerys, & Schultz, 2014) and the approaches taken in that body of literature can be pertinent here. Behavior such as spinning, finger flicking, and other repetitive, invariant motor acts often occur at a high frequency. To determine whether an intrinsically reinforcing behavior like stereotypies has operant properties, the occurrence of the stereotypies may be quantified before and after introducing competing reinforcers (e.g., food or a toy) into the environment. alternative reinforcers, depending on their reinforcing value, compete with stereotypies for time allocation and the degree to which they do depends on their relative reinforcing efficacy, as determined with preference assessments. Indeed, one theory of the development of stereotypies in animals and humans is environmental deprivation (low competing alternative reinforcers), which also tends to be correlated with lower levels of dopamine (Rapp & Vollmer, 2005).

In nonhumans, the addition of items that occasion alternative behavior also decrease stereotypies. Würbel, Chapman, and Rutland (1998) for example reported that the addition of a simple cardboard tube to the environment of mice reduced wire-gnawing stereotypies. Indeed, many studies have reported similar reductions in stereotypies when there are opportunities for other behavior (e.g., Capitanio, Kyes, & Fairbanks, 2006; Gross et al., 2012; Meagher & Mason, 2012; Meehan, Garner, & Mench, 2004; Novak, Kinsey, Jorgensen, & Hazen, 1998; Powell, Newman, McDonald, Bugenhagen, & Lewis, 2000; Wolfer et al., 2004).

Thus, stereotypies may have reinforcing properties when few or no alternative reinforcers are available. In this light, stereotypies function as just another operant that, as Young, Lawlor, Leone, Dragunow, and During (1999) suggested, fills a “behavioral void,” perhaps a better term than “boredom.” An interesting contrast is to consider animals in the wild, which spend a large portion of their time foraging or defending their territory. For example, wild pigs spend 75% of their waking time foraging (Meijer & Robbers, 2014; Stolba & Wood-Gush, 1989). When animals are placed in captivity, the time that would be allocated to foraging and defending territory is now available so the animal must do something to fill that behavioral void. In fact, some argue that doing anything has some behavioral value to the animal (Dawkins, 1983a).

Although barren environments increase the odds of stereotypies, it is important to note that stereotypies have been observed in enriched environments, too, albeit perhaps at reduced levels. For example, Gross et al. (2012) observed that ICR CD-1 mice raised in enriched environments demonstrated mainly bar mouthing on home cages whereas mice in the standard cages demonstrated a larger variety of stereotypies including jumping and backflipping from one wall of the home cage to another. The sources of this variability are unclear, but may have to do with what types of stimuli are present and available in the environment.

Restricted food schedules and adjunctive behavior

Originally described by Falk (1961, 1964) in a laboratory setting, behavior that is placed on a fixed time (or fixed interval in some situations) schedule may lead to the development of adjunctive behavior. In particular, when food or water (or some other reinforcer) is delivered at predictable intervals and independent of behavior, adjunctive behavior is observed. This adjunctive behavior has the qualities of stereotypies in that it is excessive, repetitive, and invariant. For example, if an animal receives a food pellet every 3 min, and a water sipper is present, the animal might drink enormous amounts of water (called schedule-induced polydipsia) between pellet deliveries (e.g., Falk, 1964, 1961; López-Crespo, Rodríguez, Pellón, & Flores, 2004; López-Grancha, Lopez-Crespo, Sanchez-Amate, 2008; McSweeney, Swindell, & Weatherly, 1998; Moreno & Flores, 2012; Roper, 1978). It is traditional to conceptualize adjunctive behavior as behavior that is not reinforced by programmed consequences; rather, it appears to be adventitiously reinforced. Adjunctive behavior may also have intrinsically reinforcing properties that control it. or may be an operant that emerges as a function of response–reinforcer proximity. The interested reader may examine Killeen and Pellón (2013) for a more in-depth discussion.

Adjunctive behavior has been demonstrated across many species in captivity (see Killeen & Pellón, 2013, for review), including monkeys (e.g., Schuster & Woods, 1966), pigeons (e.g., Madden & Fowler, 2011), humans (e.g., Emerson & Howard, 1992; Wieseler, Hanson, Chamberlain, & Thompson, 1988), and sows (e.g., Rushen, 1984; Terlouw, Lawrence, & Illius, 1991). It has been observed across a variety of forms in addition to polydipsia, including nose poking (Singer, Wayner, Stein, Cimino, & King, 1974), wheel running (e.g., Levitsky & Collier, 1968; McSweeney et al., 1998; White, 1985), sham chewing (e.g., Terlouw et al., 1991), and drug and saline self-administration (e.g., Grant & Johanson, 1989; Nader & Woolverton, 1992; Samson & Pfeffer, 1987), to name a few. Adjunctive behavior may be considered an “anticipatory behavior” that has Pavlovian conditioned properties (Bassett & Buchanan-Smith, 2007) or operant when there is a relatively short delay between its occurrence and reinforcer delivery (Killeen & Pellón, 2013).

Adjunctive behavior does not develop solely as a function of restricted food schedule; it may occur when other reinforcers are programmed on a fixed basis. For example, when reinforcing drug stimuli, such as cocaine (Grant & Johanson, 1989; Nader & Woolverton, 1992), sucrose (Samson & Pfeffer, 1987), or access to wheel running (Singer et al., 1974) are delivered on a time-based schedule, stereotypies indeed develop. However, because most animal care environments are devoid of such drug and alcohol alternatives, a predictable food schedule may be a likely explanation for development of stereotypies with animals in a typical captive environment.

Adjunctive behavior is well documented under short interreinforcer intervals; a 3-min interval tends to produce the most excessive behavior in laboratory studies with rats (see Falk, 1977; López-Grancha et al., 2018). Animals in captivity may have food delivered on a fixed time schedule or may have access to a reinforcer (such as social interaction), on a restricted basis but it is unlikely that the access to reinforcers would occur in such small intervals. Nonetheless, there are some reports of the development of adjunctive behavior under long interreinforcement interval schedules. For example, stereotypies such as sham chewing, chain biting, and polydipsia have been shown to occur in sows when placed under a schedule of one or two meals per day that occur at predictable times (see Douglas, Cunnick, Pekas, Zimmerman, & von Borell, 1998; Lawrence & Terlouw, 1993). Rushen (1984) found that pigs engaged in water-sipper manipulating, bar biting, polydipsia, and stereotypic head nodding when meals were served twice a day at predictable intervals, phenomena that seem to support what Staddon and Simmelhag (1971) described as terminal behavior.

In summary, in more barren environments, when food is predictable but noncontingent, behavior is susceptible to becoming locked into “anticipatory cycles” during the regularly occurring events (Jóhannesson & Ladewig, 2000). Therefore, an ideal situation would be to allow animals under human care to reside in environments that are rich with alternatives and more variable in terms of food delivery or other strong reinforcers or, in an ideal situation, arrange it so that the food is response contingent, and have an animal engage is a form of foraging for the food. Although these types of environments may be difficult to accomplish, they may benefit the welfare of the animal and prevent adjunctive behavior from occurring.

Finally, we wish to note that although noncontingent reinforcement (NCR) is relevant to the development of problem behavior like stereotypies in some animal care settings, there is an extensive literature that shows that NCR has been used successfully with humans as a tactic to reduce already-established problematic behavior that has an identified reinforcer, such as attention (see, for e.g., Carr, Severton, & Lepper, 2009; Richman, Barnard-Brak, Grubb, Bosch, & Abby, 2015; Vollmer & Borrero, 2009). A full explication of NCR’s effectiveness is beyond the scope of the present article, but the interested reader may wish to peruse some of the research on this useful technique because it could also be of value for animals under human care.

General Reduction of Behavior (Depression or Inactivity)

Another indicator of poor well-being is the observation of a general reduction of behavior that is not due to illness or disease (Broom, 1986), a condition that might be interpreted as depression. With humans, clinical depression is characterized by symptoms of diminished interest or pleasure in activities, weight gain or loss, slowing of physical movement or thought, fatigue or loss of energy, feelings of worthlessness and guilt, and recurrent thoughts of death and suicidal ideation (American Psychiatric Association [APA], 2013), all of which reflect behavioral or biological suppression. The etiology of depression in humans is frequently environmental (Rehm, LePage, & Bailey, 1994; Topic, Kröger, Vildirasova, & Huston, 2012). Adverse life events like the death of a loved one or divorce and stressors like the loss of job or trauma often result in the loss of major sources of reinforcement or can lead to a loss of reinforcement indirectly. In either case, the reinforcer loss can lead to depressive behavior or a diagnosis of depression (Huston, de S Silva, Komorowski, Schulz, & Topic, 2013; Kanter, Busch, Weeks, & Landes, 2008; Kraaij, Arensman, & Spinhoven, 2002; Lewinsohn, 1974; Pretorius, 1998). Avoidance, in the form of excessive sleeping and withdrawal from these events, may result and might even be functional in the short run, though in the long run such behavior can be debilitating. The etiology of depressive behavior in captive animals also has environmental sources so here we consider some of the determinants of depressive behavior, in animal care settings.

Learned helplessness and depression

The term “learned helplessness” describes depressive behavior that results from a behavioral history in which behavior is not effective at allowing escape or, more distressing, aversive stimuli that are delivered noncontingently. The traditional learned helplessness procedure involves the administration of unpredictable aversive events, such as shocks, over which the animal or person has no control. The initial studies (Overmier & Seligman, 1967; Seligman & Maier, 1967) were conducted with dogs that received uncontrollable and unpredictable electric shocks. When placed in an environment in which they could escape the shocks by engaging in some response, they failed to do so. This learning deficit was often accompanied by a general suppression of behavior as well (Seligman & Maier, 1967). It is remarkable that the inability to control shock also increased ulceration and susceptibility to cancer (see Maier & Seligman, 1976; Mineka & Hendersen, 1985, for reviews). Learned helplessness is associated with changes in the hypothalamic–pituitary–adrenal axis, a physiological system well-known to be linked to a constellation of stress responses (Chover-Gonzalez, Jessop, Tejedor-Real, Gibert-Rahola, and Harbuz, 2000; Takamori, Kawashima, Chaki, Nakazato, & Kameo, 2001). It is important to note that once animals were able to terminate the shocks with successful escapes, their activity levels returned to their original nondepressed baseline (Seligman & Maier, 1967; Solomon & Wynne, 1953).

The phenomenon of learned helplessness has been observed with many species, including rats (e.g., Maudhuit et al., 1997), monkeys (e.g., Rush, Mineka, & Suomi, 1983), cats (e.g., Thomas & DeWald, 1977), elephants (Derby, 2008; Moussaieff Masson & McCarthy, 1995), gorillas (Wells, 2005), and humans (Teodorescu & Erev, 2014). Numerous studies have been conducted on this phenomenon, and its effects are robust and predictable. The phenomenon has been generalized to other situations in which the animal has no control over important events so learned helplessness appears even when what is delivered would, under other circumstances, would serve as a positive reinforcer (e.g., Ferrándiz & Pardo, 1990; Job, 1988; Oakes, Rosenblum, & Fox, 1982).

Extinction and depression

The process of extinction requires first that behavior is maintained under some schedule of reinforcement and then the reinforcers are removed, making this a transition from an enriched environment to a more impoverished one (in a technical sense extinction can also involve a transition from response-contingent to noncontingent consequences but that is a less common usage). The behavior pattern under extinction is well-documented and predictable: when an established source of reinforcement is removed, behavior (e.g., lever-pressing) initially increases in rate, variability, force, or other measures of behavior that has previously led to reinforcement (Neuringer, Kornell, & Olufs, 2001; Reed & Morgan, 2006). This collection of behavior is termed an “extinction burst” (Pierce & Cheney, 2018). Eventually behavior declines to near-zero levels (Ferster & Skinner, 1957; Podlesnik & Shahan, 2010).

The behavioral effects of extinction have been compared to depression (e.g., Ferster, 1973; Huston et al., 2013) and the extinction burst has been interpreted as a sign of frustration (Amsel, 1958). Indeed, animals behave as though the loss is upsetting (Amsel, 1958), and then the burst of the behavior eventually weakens until little behavior is observed; the attribution of frustration is an understandable anthropomorphism. This predictable characteristic of extinction also has been described as an animal analog of depression (Ferster, 1973; Kanter, Cautilli, Busch, & Baruch, 2005; Klinger, Barta, & Kemble, 1974) in which the characteristic inactivity is interpreted as a depressive episode. When the reestablishment of behavior is especially insensitive to subsequent reinforcement (similar to learned helplessness), the parallels between extinction and depression seem similar (Ferrándiz & Pardo, 1990; Job, 1988; Oakes et al., 1982). Whether the diagnosis of “depression” is necessary is of less consequence than the behavior patterns that occur in the learned-helplessness or extinction situations. These patterns can be signs of trouble.

A related literature is worthy of mention. Some reports state that infant nonhuman primates that undergo social separation from their mothers (a major source of reinforcement that is attachment-based or socially mediated) may develop a behavioral pattern that is referred to in clinical literature as the biphasic protest–despair response (Mineka & Zinbarg, 1991). At first, the infant monkey behaves with “intense agitation and distress” sequalae (Mineka & Zinbarg, 1991, p. 64) that may look similar to what Amsel (1958) reported when extinction is first placed in effect. If the separation continues for days, the infant monkey recesses into what may be called a depressive state, in which the monkey is behaviorally withdrawn and inactive.

The biphasic protest–despair response has been documented in a wide range of monkeys, including pigtail, rhesus, squirrel, Java, and Indian langur (Chappell & Meier, 1975; Dettling, Feldon, & Pryce, 2002; Dolhinow, 1980; Hinde & McGinnis, 1977; Kaufman & Rosenblum, 1967; Levine & Mody, 2003; Reite, Kaemingk, & Boccia, 1989; Spinelli et al., 2007), as well as other mammals including rats (Hofer, 1994, apes (Codner & Nadler, 1984), and chimpanzees (see Mineka, 1982, for review). This pattern also has been observed in primates following the removal or death of a conspecific of which the animal spent most of its time (see Bales et al., 2017; Mineka & Suomi, 1978; Washburn & Rumbaugh, 1991, for reviews).

There exist important similarities between responding under extinction and the biphasic protest response. First, the removal of an established reinforcer is involved, whether it is food, a conspecific, or other reinforcer. Second, there is a characteristic and immediate increase in behavior and variability in behavior, often occasioning attributions of frustration or other emotional responses. Third, following that, behavior tends to decline. In some cases, the effects may be long lasting, if not irreversible. There are also important distinctions between these two phenomena, however. For example, one is relevant to loss of attachment, a complex biological and social phenomenon that has profound and long-lasting effects, and the other with more short-term losses, such as food pellets. Nonetheless, it is worth drawing attention to the similarities of behavior under extinction and the biphasic protest response. Indeed, the biphasic protest may be a profound behavioral effect of extinction.

Displaced Aggression

Aggression in the wild with nonhumans is a frequent response to such environmental situations as territorial intrusions and competition for food or mates that are mediated and even amplified by endocrine changes (Bhatt, Zalcman, Hassanain, & Siegel, 2005; Giammanco, Tabacchi, Giammanco, Majo, & Guardia, 2005; Owen-Ashley & Butler, 2004; Scott, 1958; Siegel & Victoroff, 2009). These changes in aggression and the endocrine system also occur during breeding season in animals that are season breeders, which includes most mammals and avian species (Soma, Scotti, Newman, Charlier, & Demas, 2008). The social stress that occurs during breeding season may be related to dominance expression, which is linked to reproductive success (Creel, 2005). Indeed, aggression in the wild has beneficial effects and survival value for these reasons.

Aggression also occurs in captive environments, usually in forms that appear inappropriate and functionless, because territory is available and competition for resources is irrelevant. This behavior, referred to as displaced aggression, has causes that are environmental as well. In captive situations, when at least two animals are pair-housed in a relatively small space, an aversive stimulus, such as a shock, triggers what looks like a reflexive, aggressive response toward a conspecific or object (Azrin, Hutchinson, & McLaughlin, 1965a, 1965b). A classic series of experiments by Ulrich and Azrin (1962), for example, reported on rat pairs placed under a schedule of unpredictable shocks that varied between 0.1 to 38 times per min. An attack response appeared immediately after shock was presented and the number of attacks was related strongly and linearly to shock rate. This phenomenon has been demonstrated across other species, including hamsters, guinea pigs, and nonhuman primates (Azrin, Hutchinson, & Hake, 1963; Azrin et al., 1965a, 1965b; Hutchinson, Azrin, & Renfrew, 1968b). Ulrich and Azrin (1962) also replicated the shock-induced fighting response across other aversive stimuli, including shock by electrode and intense heat; aggression was not observed, however, when the aversive stimulus was dry ice or noise.

Aggressive behavior does not require the presentation of primary aversive stimuli. Extinction, which also has aversive properties, can provoke an animal to attack a nearby conspecific when a stimulus correlated with extinction is presented (Azrin, Hutchinson, & Hake, 1966). The aversive properties of extinction are functionally similar to that of the presentation of other aversive stimuli. In one study, for example, food was made available to pigeons for 5 min, and then was withheld. Aggressive behavior (pecks to the throat, head, eyes of a target pigeon) occurred within 30–60 s of the onset of extinction. This effect was demonstrated whether extinction was signaled or not by a stimulus and depended on whether the bird consumed the food before extinction was placed in effect. Many of the early studies conducted on extinction-induced aggression used pigeons (Looney & Cohen, 1982), but the phenomenon has been reported in rats (e.g., Huston et al., 2013; Thompson & Bloom, 1966), monkeys (e.g., Emley, Hutchinson, & Brennan, 1970), hens (Haskell, Coerse, Taylor, & McCorquodale, 2004), and humans (e.g., Frederiksen & Peterson, 1974; Harrell, 1972; Kelly & Hake, 1970), suggesting that this is a general phenomenon. (See Lerman, Iwata, & Wallace, 1999, for a review of its prevalence in applied settings and Miczek, Fish, de Bold, & de Almeida, 2002, for a review of the importance of animal studies in this area in the psychopharmacology of aggression.)

Enclosures that are too small can induce aggression and increase cortisol levels, a marker of stress and activation of the HPA axis, if animals are pair- or group-housed (e.g., Carlstead, Mellen, & Kleiman, 1999; Flauger & Krueger, 2013; Li, Jiang, Tang, & Zeng, 2007). This is especially evident when organisms are transferred from large enclosures or from the wild into small enclosures. Although this type of aggression is related to territory or resources, it may also relate to a transition from a richer to a leaner schedule of reinforcement.

Studies of the conditions promoting aggressive behavior have implications for the housing of animals in captivity. First, although some species in general interact well with conspecifics (e.g., younger rats, monkeys, and pigeons), environmental stressors (e.g., noise, heat, food deprivation, coarse handling of animals or cages) can create situations in which aggression may occur. Second, cage sizes designed for animals are based on empirical studies that rely on measures of stress, such as the presence of stress hormone and changes in heart rate (e.g., Li et al., 2007; Line et al., 1989), but in many situations these tests are conducted in the absence of aversive stimuli. Perhaps more space might be considered for situations in which two or more animals are housed together (the standard in the United States) and when aversive stimuli may unexpectedly enter the environment.

As a side note, the study of aggression represents an interesting example of the difficulties in conducting research of behavior under aversive control. It is unlikely that many of the seminal studies in this area, some cited above, would be performed today because of a wider appreciation of issues surrounding pain and distress in laboratory animals, an issue captured in the 1985 amendment to the AWA and endorsed by ethical guidelines for psychologists (e.g., APA, 2017). Although this has certainly prevented some unnecessarily distressful experiments from being conducted on nonhuman animals, it may also have suppressed the conduct of carefully designed studies, thereby limited our understanding of behavior under aversive control, including aggression, pain, anxiety, or stress (Critchfield & Rasmussen, 2007). Where such work has continued to be conducted, it has contributed to our understanding of the importance of housing conditions, space, and endocrine activity in animal well-being and aggression (e.g., see Miczek et al., 2002; Verwer, van der Ven, van den Bos, & Hendriksen, 2007).

The “Interactive Environments are Beneficial” Approach

When comparing animals in captivity to animals in the wild, there are disparities in opportunities for environmental interaction. This absence of an interactive environment may be what stirs sympathy from humans when they observe animals in small, sparse enclosures. Although the animals might show no overt signs of distress or mistreatment—behavioral or otherwise—their placement in a barren environment, devoid of alternatives, occasions one to ponder if indeed they are “happy.”

To show that an animal is psychologically well requires behavioral assessment, either in terms of careful observation or by assessing responses to carefully selected challenges. As a start, it might be useful to examine whether the opportunity to manipulate its own environment is valuable to the animal; in other words, to what extent is there a place for operant behavior in the PWB of animals and will the animal choose/prefer such an opportunity?

Four areas in the experimental analysis of behavior and experimental psychology literature can be harnessed to assess the value of engagement in operant behavior. First, animals can select from among differential outcomes (i.e., they exhibit preferences for specific stimuli and conditions) and this phenomenon can be used to assess what outcomes are preferred. Such a quantification of reinforcer value is critical to considerations of animal PWB. Second, in a barren environment operant behavior can produce variability itself. Therefore, the extent to which variability is preferred, and to what extent it has behavioral and biological benefits should be considered. Third, operants provide control over the environment. An assessment of the extent to which control is a preferred feature of an environment for animals under human care can be determined. Finally, in barren conditions, animals prefer to obtain reinforcers through making effortful responses, as opposed to their noncontingent delivery. Each of these areas is addressed below.

Quantifying Reinforcer Value

Recognizing that an animal can demonstrate preferences for some reinforcers over others leads quickly to tactics for quantifying what is enriching or aversive to that animal. Quantification and careful experimentation remove the guesswork and anthropomorphizing involved with subjective hypotheses about what is beneficial to an animal.

The quantification of reinforcer efficacy has been detailed in numerous papers (e.g., Hursh & Silberberg, 2008; Trosclair-Lasserre, Lerman, Call, Addison, & Kodak, 2008). Three basic procedures, each offering different types of information, have been used to quantify reinforcer value for individual animals, as well as species on a general level. These procedures include simple preference tests, choice, and behavioral economic measures that use effort to quantify value.

Simple preference tests

Because simple preference tests vary so widely in their format, an exhaustive literature review of their nuances would be beyond the scope of this article, so we will describe them more generally. In a preference test, an animal is presented with access to two or more reinforcers in a space for a period of time. The reinforcer with the higher time allocation (beyond chance) is regarded as the preferred alternative. Another assumption is that if an animal chooses a particular alternative exclusively, then it strongly prefers it. Van de Weerd, Van Loo, Van Zutphen, Koolhaas, Baumans (1997), for example, used preference tests to determine the types of nesting materials (paper towels, Kleenex, or an empty box) that mice prefer in their home cages. Mice overwhelmingly chose the nest box with the paper towel or Kleenex. The authors noted that in some instances during testing, mice would drag the preferred nesting materials into the other parts of the apparatus. Van de Weerd and Baumans (1999) showed that mice prefer wood shavings and paper particles over plain wire mesh. (It should be noted that wire mesh is no longer recommended as a default suitable flooring in most research environments in the United States.) The authors stated that because the mice moved and manipulated these types of materials, that manipulation might be a property that is important for floor bedding or nesting. The authors also asserted that this observation supports the idea that control is valued, a statement that is explored elsewhere in this article. Preference has been examined for not only bedding (Kawakami et al., 2007), but also for ambient lighting and cage size (e.g., Sherwin & Sherwin, 2007; Jochems, van der Valk, Stafleu, & Baumans, 2002).

Preference tests have been used in farm animal welfare to assess cage size (e.g., Faure, 1986; Dawkins, 1983b), food types (e.g., Early & Provenza, 1998; Villalba & Provenza, 1997), acoustic stimuli (e.g., MacKenzie, Foster, & Temple, 1993), and conspecific access (e.g., Dawkins, 1982).

Simple preference tests can also assess the aversiveness of stimuli by quantifying escape or avoidance. Research with responses to noisy environments is an illustrative example. MacKenzie et al. (1993) determined what types of auditory stimuli (e.g., barking dogs or fighting hens), as well as their intensities, are aversive and appetitive for battery hens. Hens could move to one side of their cages to terminate the sound or to the other side to continue the sound. Hens preferred the sound of other hens in a poultry shed at the 70-dB level, for example, but at the 80- and 90-dB levels, indifference was apparent. This showed that intensity of a sound could be a factor in what is considered aversive, but so is the type of sound. Operant techniques like these also have been used to determine aversive noise thresholds for hens (Owen, Swaisgood, Czekala, & Lindburg, 2005; Temple, Foster, & O'Donnell, 1984).

The strengths of these approaches are that they are relatively simple to implement with little investment in resources and can be initiated quickly but they have some limitations. It is sometimes unclear whether the less frequently chosen alternative is actually aversive, and therefore avoided, or whether the more frequently chosen alternative is more relatively preferred. Sometimes a none-of the-above option may assist in interpretation but this dilemma also taps a general quandary about difficulties in distinguishing avoidance from reinforcement (see Perone, 2003). For example, is food reinforcing or is the discomfort associated with hunger avoided? This issue does have practical implications when interpreting preference assessments, in part because of the regulatory environment under which they are sometimes conduced. Time-based aspects of reinforcer consumption may confound the ability to compare relative durations of time spent consuming qualitatively different reinforcers. The consumption of some reinforcers takes more time than others; consuming food, social interactions, or mating all may take different amounts of time and these can complicate the assessment of relative time spent with an alternative. Another limitation is that preferences are dynamic and change with experience or satiation or can be sensitive to the cost of obtaining them. Other measures of reinforcer efficacy may help circumvent these problems.

Choice, concurrent schedules, and the matching relation

One interpretation of “well” could be that an animal is distributing its behavior among many possible environmental alternatives and freely choosing its own reinforcers. The concept of an animal allocating behavior to multiple sources of reinforcement in proportion to how much is available.

In laboratory studies, choice and value have been described quantitatively using the matching relation, which in its simplest form postulates that, given two alternatives, the relative rate at which one responds to receive one alternative, or the time spent with that alternative, is proportional to the relative rate of reinforcement that the response produces (Herrnstein, 1961, 1970). An animal will allocate more behavior, or time (Baum & Rachlin, 1969) toward an alternative that has delivered reinforcers more frequently in the past than another so this allocation quantifies value. Rarely is the allocation of behavior all-or-nothing: preference is usually graded. The use of a concurrent schedule of reinforcement is the basic choice procedure because it is sensitive to even small differences in reinforcer rate, quality, or amount (deVilliers, 1977; Davison & McCarthy, 1988).

The prototypical example is the concurrent variable interval-variable interval (conc VI VI) schedule, in which a response (e.g., a lever press) on one of two options produces a reinforcer after a varied period of time has passed. Responses on the other lever produce a reinforcer after a different amount of time elapses. If, for example, if the richer alternative produces 2 reinforcers/min and the leaner one produces 1 reinforcer/min then 2/3 of the responses or time (2/(1+2)) should be allocated to the richer alternative. In fact, the behavior allocation nearly always undermatches the allocation of reinforcer densities (Baum, 1979). In our example, would allocate slightly fewer than twice as many responses to the richer alternative than to the leaner one, demonstrating a preference for the alternative that delivers more reinforcers; see, e.g., Herrnstein, 1970; Kalenscher & Van Wingerden, 2011; Sakai & Fukai, 2008).

The generalized matching relation parses two sources of deviation form matching—bias and sensitivity to the relative reinforcer densities (see Baum, 1973). The degree to which preference undermatches is a measure of sensitivity to the reinforcer differences. If there is no preference then there is no sensitivity to any distinctions between the two outcomes. Bias is a consistent tendency toward one alternative regardless of the rate at which it is delivered. Bias could be especially relevant to questions of well-being because it is a quantitative indicator of overall preference for one reinforcer over another, regardless of the relative reinforcer densities. Indeed, a number of laboratory studies have demonstrated the utility of the bias parameter as a measure of preference (Bron, Sumpter, Foster, & Temple, 2003; Buckley & Rasmussen, 2012, 2014; Miller, 1976; Rashotte & Smith, 1984). For example, a quantitative metric of the degree to which pigeons and rats prefer different grains or pellet flavors was determined using hedonic scaling techniques built upon precise, matching-based measures of bias for one flavor over another (Buckley & Rasmussen, 2012; Miller, 1976).

Generalized matching has been underutilized in studies related to the PWB of animals. This is likely because the methodological rigor often required for experimental studies using concurrent schedules of reinforcement is simply not feasible in an animal care setting. For example, isolation of conditions, the use of longer sessions, criteria and time for stability to be reached, and the use of multiple ratios of reinforcement may be difficult to implement. However, sometimes simple approaches with modest equipment requirements are sufficient. Baum and Rachlin (1969) place pigeons in a long enclosure in which simply standing on one end resulted in food deliveries at one rate and standing on the other at a different rate. Time allocation to the different alternatives matched reinforcer deliveries, indicating the value of those locations.

We include a discussion of concurrent schedules because the matching relation, which is a highly reproducible phenomenon that appears almost instantaneously (Baum & Rachlin, 1969; Gallistel et al., 2007), could be applied in natural or experimental settings by conceptualizing all of the possible alternatives an animal has available at a given time and examining the allocation of its behavior toward any of those reinforcers. Many parameters, aside from frequency of payoff, figure into what an animal chooses. Indeed, the value of a reinforcer is multivaried and includes rate, amount, and immediacy of reinforcement (e.g., Baum & Rachlin, 1969; Hackenberg, 2009), quality of reinforcer (e.g., Foster, Temple, Robertson, Nair, & Poling, 1996; Klopfer, Ruakura, Kilgour, & Matthews, 1981; Matthews & Temple, 1979; Calvert, Green, & Myerson, 2010), topography of responses (e.g., lever press vs. wheel run) required to produce the reinforcer (e.g., Allan & Zeigler, 1994; Sumpter, Foster, & Temple, 1995), and possible aversive properties of an outcome (e.g., Baum, 1973; Deluty, 1978; Zentall & Singer, 2007; Pessiglione & Delgado, 2015). Sometimes, it may be beneficial to model the impact of multiple alternatives, which is more likely to be encountered in the real world, and techniques for doing so have been described (Jensen & Neuringer, 2009).

In summary, we believe that when it comes to choice and preference, research questions that are aligned with quick assessment for animals (e.g., degree to which an animal prefers a blanket in its sleeping quarters) may benefit from using simple preference tests. The matching relation and the procedures that enable its assessment are an extension of these preference tests and it provides a principle-based, highly quantitative assessment of value with the rigor seen in a laboratory study. With creativity, the range of questions that can be addressed is expanded. For example, is manipulation a preferred property of bedding in rodent species? This issue is addressed in more detail below when discussing concurrent chains. Indeed, the choice procedures described above have utility for a large range of research questions related to PWB and make it clear that animals demonstrate preferences for some features of their environment over others.

Effort: Progressive ratio schedules and consumer demand

A third way to examine the strength or value of a reinforcer is to determine to what extent an animal will exert effort to produce it. The progressive ratio (PR) schedule of reinforcement is one procedure that examines effort-based motivation for specific outcomes (Stafford, LeSage, & Glowa, 1998). Under this schedule, an animal is required to perform an arbitrary response, such as pressing a lever, for access to a commodity. The initial response requirement is small, 1–5 responses, but with each completed ratio, the value of subsequent response requirement increases by a certain percentage or amount within the session (see Killeen, 2014; Killeen, Posadas-Sanchez, Johansen, & Thrailkill, 2009; Rickard, Body, Zhang, Bradshaw, & Szabadi, 2009). Eventually the animal reaches a “break point” at which it no longer responds. The break point is a measure of the commodity’s value. Indeed, the value of different types of reinforcers, including food (Covarrubias & Aparicio, 2008; Poling, 2010; Spear & Katz, 1991) and drugs (e.g., Donny et al., 1999; Stafford et al., 1998) have been compared using breakpoints under PR schedules.

A related area applies economic principles of consumer demand theory (see Rodefer & Carroll, 1997, for a direct comparison). In brief, effort-based price is manipulated between sessions, typically as fixed ratio schedules (as opposed to within-session changes used with PR), so many reinforcers can be earned in a session at the same price. “Demand” is the relation between the number of reinforcers consumed and effort-based price (Bickel & Vuchnich, 2000; Hursh, 1980; Hursh & Silberberg, 2008). The lower the price, the more reinforcers are earned and as price increases the number of reinforcers decreases. In terms of methodology, good practice requires that the commodity being assessed be available only during experimental sessions, a procedure called a “closed economy” (Madden, 2000).

Elasticity is quantified by plotting reinforcers earned against the price/reinforcer (unit price) on a log–log scale (Hursh & Silberberg, 2008). The function that describe demand curves accelerates downward. When the commodity is cheap, a small price increase has little impact; doubling the price reduced demand by less than half and this is termed inelastic demand. There comes a point, however, when small price increases cause large decreases in consumption; doubling the price reduced consumption by more than half and demand is said to be elastic. There is a point at which a doubling of price results in an exactly halving of consumption. This point of unit elasticity, sometimes called P_max because it represents the price at which consumption is at its maximum, is an important marker of value and can be used to compare commodities in some approaches. When the price is very low preferences for the different commodities are sometimes indistinguishable. It is only when the price increases that the reinforcers can be distinguished so demand functions can reveal stable preferences that may be invisible to simple preference assessments conducted with a minimal response cost. The different approaches available for analyzing demand curves are summarized in Hursh and Silberberg (2008) and Reed, Niileksela, and Kaplan (2013) has an excellent tutorial.

Demand curves have been generated for different reinforcers pertinent to animal welfare. Matthews and Ladewig (1994), for example, determined elasticity of three reinforcers for pigs’ behavior: food, an empty space, and that same space with a littermate placed there. The response was pressing a plate that lifted a hinged door. Food demand was inelastic whereas social contact demand was more elastic. Demand for access to an empty space was the most elastic. Closer inspection indicated that demand for food was inelastic over the entire range of response requirements and that access to empty space was elastic, and even small price increases resulted in large reductions in responses. These observations permit a clear, quantitative, and objective comparison of the importance of these three commodities to the pigs and did so only by continuing to increase the response requirement for each one until the animals stopped accessing the reinforcer. Such information could not be obtained with simple choice procedures and would have been difficult to obtain through application of the matching law. The observation that food is a primary reinforcer, and has such high necessity to organisms may be a good anchor when comparing the value of other commodities with demand curves (Dawkins, 1990).

Dawkins (1990) reported that the use of demand functions in assessing commodity importance gained acceptance within the animal welfare community because of its ability to quantify preference (e.g., Dawkins, 1983a, 1983c; Foster, Temple, Cameron, & Poling, 1997; Lee, Floyd, Erb, & Houpt, 2011; Sherwin & Nicol, 1995). Further, she noted that this method provides a way to create a “shopping list” of alternatives in order of importance. Dawkins (1990) also referred to the inelastic commodities as those that are biological needs. Although more preferred items tend to be those that serve a biological function that is necessary for sustenance, such as food, water, social interaction (for some species), it is important to note that it is not necessary to refer to these items as needs, but only as those in which the animal will respond at high prices under certain condition. For example, cocaine is a reinforcer that also yields inelastic demand functions (e.g., Koffarnus & Woods, 2013; Mattox & Carroll, 1996), but few would argue that it is a biological need.

Comments

Allowing animals to have control of their surroundings is a useful and objective method to determine what is valued by an animal. In many cases people assume that what a human prefers is also preferred by the animal. For example, one may assume that the larger an animal cage is, the more beneficial it is to the animal, but a close examination reveals the hazards in anthropomorphism. A study by Faure (1986) in which hens were trained to peck a key that expanded their cages, demonstrated results contrary to this hypothesis. The hens indeed pecked to maintain a larger cage size, but it was not the full capacity of the cage. When making decisions about enrichment, the choices are often best left to the animals. The above-mentioned procedures provide valuable empirical and theoretical implications for assessing the things that are important, or even essential, to an animal in captivity. Because animals exhibit preferences for certain objects or husbandry supplies, it can be said that those have value to the animal. It is worth noting, however, that more alternatives may not necessarily be better. Although a moderate number of alternatives in some instances has led to clear preferences being demonstrated, too many alternatives may be aversive to the organism (see Hutchinson, 2005, for a critical review of the literature in this area).

Variability and Change

Two general areas contribute to the notion that variability is important to PWB. These include preference for varied schedules over fixed and the behavioral and biological effects of environmental enrichment.

Preference for variable over fixed schedules of reinforcement

The concurrent chain laboratory procedure can be used to characterize animals’ preferences for complex outcomes, including variable schedules of reinforcement over fixed schedules (see Davison, 1969; Fantino, 2008). It offers a choice between two initial links, the completion of which produces access to one of two terminal links. For example, the initial links may be a simple fixed ratio 1 (FR1) schedule, in which one lever press on one of two levers produces access to the correlated terminal link. (It is sometimes not this simple, but such details will not be discussed here.) The more important variable, in this context, is the terminal link. To study preferences for predictable versus varied environments, for example, one alternative will lead to a fixed interval schedule and the other to a variable interval schedule of food delivery. In another implementation the animal may choose between food that is or is not response dependent. A large literature demonstrates the validity of this procedure: the animal’s choices for a particular alternative increase as the reinforcer density in the terminal link increases, corresponding to the matching relation described earlier (e.g., Herrnstein, 1970; Kalenscher & Van Wingerden, 2011; Shahan & Podlesnik, 2006; Sakai & Fukai, 2008) and to preferences for reinforcers presented sooner over those presented later (Fantino, 1977; Rachlin & Green, 1972). This supports the requirement that the concurrent chain schedule reveals the sort of preferences that are characterized in the experimental literature using other techniques.

The concurrent chain procedure has been invaluable in evaluating animals’ preferences for reinforcers with predictable versus varied outcomes, and some of the results may be surprising. In one procedure (Kendall, 1974), one initial link leads to a reinforcer following the completion of the terminal link schedule 100% of the time. For the other alternative, a reinforcer is unpredictable and occurs only 50% of the time. The overall reinforcer rates or magnitudes are identical for the two terminal links. Kendall (1974), for example, showed that pigeons preferred the initial link associated with 50% food reinforcement more often than one that predicted food reinforcement 100% of the time. The preference for unreliable reinforcement over reliable reinforcement has been replicated in many other studies with a variety of options (e.g., Belke & Spetch, 1994; Dunn & Spetch, 1990; Gipson, Alessandri, Miller, & Zentall, 2009; Kendall, 1985; Smith & Zentall, 2016).

In addition to assessing preference for varying reinforcer probability, the predictability of the terminal link's duration has also been investigated. Studies on preference between fixed versus variable versus mixed duration (fixed and variable) of terminal links, for example, have revealed that animals prefer terminal links that vary in duration (e.g., Foster et al., 1997; Grace & Nevin, 2000; Grace, Bedell, & Nevin, 2002; Spetch, Belke, Barnet, Dunn, & Pierce, 1990). Cicerone (1976), for example, programmed terminal links that delivered reinforcers according to either a fixed delay interval (always 0, 8, 16, or 32 s) or a varied basis (2 or 14 s in some instances or 6 or 10 s in others). When the fixed time delay in one choice was 0 s, subjects chose the fixed time terminal link between only 17% and 36% of the time, which is noteworthy because an immediate reinforcer would seem to be preferred to one that is not immediate. When the other fixed delays were examined, preference for the varied schedule increased even more and did so in a delay-dependent fashion; that is, as the fixed delay increased, preference for the varied or mixed delay increased. Preference has also been shown for terminal links that require a varied number of responses to produce the reinforcer over a fixed number of responses (e.g., Duncan & Fantino, 1972; Field, Tonneau, Ahearn, & Hineline, 1996; Xi et al., 2005).

The preference for variability has much to do with the probability and delay of reinforcement because an animal may prefer a variable schedule because sometimes the reinforcer arrives earlier than on a fixed schedule with an identical probability of food. The observation that variation is chosen over predictable outcomes suggests that there is something about variability in and of itself that has a reinforcing property to captive animal behavior. The mechanism may be the very short delays to reinforcement that sometimes occur; this may be important to setup for captive animal environments

Environmental enrichment (EE)

We have already noted that opportunities to engage in environmental interaction decrease stereotypies and increase activity in captive environments. In a varying environment, environmental complexity also has many neurobiological benefits. Enriched environments compared to impoverished ones can lead to greater synaptic branching, more complex neural pathway development, larger brain mass, and may stimulate the growth of new connections (e.g., Fabel et al., 2009; Komitova, Mattsson, Johansson, & Eriksson, 2005; Laviola, Hannan, Macrì, Solinas, & Jaber, 2008; Lewis, 2004; Nithianantharajah & Hannan, 2006; Rosenzweig & Bennett, 1978).

EE reduces the impact of degenerative conditions on the brain and even exposure to environmental contaminants. Indeed, experimental studies with animal models show that EE or the absence of stressors buffers the neural effects of challenges associated with traumatic injury, Alzheimer’s disease, Parkinson’s disease, and stroke (e.g., Costa et al., 2007; Faverjon et al., 2002; Janssen et al., 2010; Kleim, Jones, & Schallert, 2003; Norena, 2005; van Rijzingen, Gispen, & Spruijt, 1997). Enrichment also may reduce the neural decline associated with aging, anxiety, and high-fat diet (e.g., Bhagya, Srikumar, Veena, & Rao, 2017; Faherty, Raviie Shepherd, Herasimtschuk, & Smeyne, 2005; Maesako et al., 2012a, 2012b; Moncek, Duncko, Johansson, & Jezova, 2004; Mora, Segovia, & del Arco, 2007; Urakawa et al., 2013). EE or the absence of stressors can also ameliorate the effects of exposure to neurotoxicants (e.g., Faherty et al., 2005; Tyler & Allan, 2013; Virgolini et al., 2008).

An animal care environment will not be as plentiful in sources of stimulating events as the natural environment. There may be fewer opportunities to demonstrate “wild” motor behavior such as running, foraging, or copulating. There are also fewer sounds, odors, and sights. Indeed, a standard laboratory setup is referred to as an impoverished environment in research in studies that examine environmental enrichment (e.g., Balcombe, 2006; Rosenzweig & Bennett, 1978; Špinka & Wemelsfelder, 2011). As such, EE is worthy of consideration for a wide range of animal care settings. An example of the successful application of EE with animals under human care in the United States and worldwide is with zoos. Indeed, the Association of Zoos and Aquariums specify the formal development and implementation of EE programs for accredited zoos. To the interested reader, there is a wealth of literature of these programs, including discussions on evaluation of effectiveness (e.g., Alligood & Leighty, 2015; Kagan et al., 2015; Whitham & Wielebnowski, 2013).

Operants Provide Control over the Environment

Ethologists have argued that control over the environment is one of the most essential aspects of PWB (see Bassett & Buchanan-Smith, 2007; Chamove, 1998; Wemelsfeder, 1990) and here we describe experimental evidence for this statement. The rationale is that in the wild, an animal has a great degree of control over what it approaches and what it may avoid from moment to moment, and this control is removed when placed in captivity (Chamove, 1998). For example, animals under human care may be unable to escape from aversive stimuli, such as ambient noise, because their environment does not allow them to do so; this may cause an aberrant change in the animal’s behavior. It is also argued that an animal that has control over the environment is able to produce the relevant coping responses in stressful situations that enable it to survive (Chamove, 1998).

Owen et al. (2005) describe the importance of allowing zoo animals the choice of where to spend their time. When Giant Pandas are allowed to choose between an exhibit room or their sleeping quarters, they demonstrated fewer signs of agitation and also had lower levels of cortisol in their urine. This occurred even though the sleeping quarters were less enriched than the exhibit enclosure, showing that escape from stimulation is sometimes a preferred outcome. Allowing the pandas the ability to escape possible stressors, such as visitors, bright lights, and noise, improved well-being of the animals. That the Giant Pandas did not spend all of their time in one area or the other recalls one of the messages of the matching relation. Choice is rarely discrete or all-or-none. Instead, there can be graded preferences that change from time to time.

One property of operant behavior is control over the types of consequences that an animal produces via behavior. Although consequences are defined as increasing or decreasing the probability of behavior, the fact that a response must occur for a consequence to be delivered, i.e., behavior controls consequences, may be overlooked. The animal may be viewed as choosing the events to produce or avoid, and this choice is based in what has been reinforcing or punishing in the past. The importance of control over aversive events has already been discussed with regard to learned helplessness and aggression, but there also seems to be some support from the literature in behavior analysis and EE that suggests that control of appetitive stimuli may be important to well-being.

One essential feature of control is contingency—the cause-and-effect relation between behavior and outcome. The animal operates on its environment that contingently produces a specified outcome. A rat’s bar press produces a food pellet, for example, and certain motor behavior may result in access to sleeping spaces or an exhibit room. Receiving a noncontingent food pellet reduces control over its delivery. Programming the environment in such a way that an animal receives reinforcers independent of behavior places the animal at risk for adjunctive (or superstitious) behavior, as previously described (e.g., Killeen, 2014; Skinner, 1948; Staddon & Simmelhag, 1971). In a barren environment, in which few alternatives exist, a fixed time schedule of (noncontingent) reinforcement may adventitiously reinforce stereotypies, including pacing or rearing immediately before mealtime. This is certainly relevant to animals under human care because they are often fed on predictable, noncontingent schedules. The question that remains, however, is whether this type of schedule is harmful.

Control over the availability of appetitive stimuli has been compared to freedom because it might be viewed as choice behaviors that are controlled by positive reinforcers, as opposed to aversive contingencies (Skinner, 1971). Catania and Sagvolden (1980) argued that the true test of this conceptualization of freedom is whether an organism will prefer a free choice over a forced one. In other words, will an animal prefer a choice that is forced by the experimenter or one that it has control over, even when the outcome (a reinforcer) is the same? To examine this, Catania and Sagvolden (1980) used a concurrent chain schedule in which pigeons could select a terminal link that provided a choice of an array of response devices to peck or a terminal link that provided only one response device to peck. Pigeons consistently preferred the alternative that allowed them to choose from an array of response devices, even though the probability for reinforcement for both options was identical. Other studies (Catania, 1975; Ono, 2004) have shown similar results.

Control over the environment is not only preferred, it may also be beneficial. For example, animals that have access to contingent reinforcers (e.g., food) tend to perform better on learning tasks involving both appetitive and aversive stimuli than animals who receive reinforcers on a time-only basis (see Badihi, 2006; Buchanan-Smith, 2011; Overmier, Overmier, Patterson, Wielkiewicz, 1980). Further, animals that control their own food delivery are in general healthier and more behaviorally exploratory (Markowitz & Woodworth, 1978; Meehan & Mench, 2007). This area can easily be viewed as an extension of the environmental enrichment literature because a hallmark of enrichment is control over what toys to play with, what bedding to use, and what conspecifics to interact with.

Choosing Effort

We have argued that the opportunity to engage in operant behavior has measureable, quantifiable value to animals, and such opportunities may affect PWB. One true test of the value of engaging in operants would be to provide an animal with two means of receiving a reinforcer: one in which a response (such as a bar press) is required, and one in which the animal receives the reinforcer independent of the response (food is “freely” available). Observing the extent to which the animals chooses the response-dependent option indicates the value of performing the operant. Note that there is still a response required to obtain “free” food, such as moving toward or being near the “free” food dispenser, but it is minimal. The response-dependent option involves moving toward or being near the contingent-food dispenser, plus an additional response, e.g., a bar press or several bar presses, that produce the food. Also of importance, the reinforcer in the “free” food condition involves more than one food pellet at a time per response, whereas in the response-dependent alternative, usually only one pellet is delivered per response, whatever the requirement may be. All else being equal, it seems intuitive that the matching relation should predict more allocation toward the response-independent source because more reinforcers are possible and less effort is required. Indeed, complete preference for the free food might be predicted

This is not the case, however. It is interesting that, in many cases, animals in laboratory settings show greater than 50% preference for the response-dependent food source, and in some cases show greater than 75% preference, a phenomenon called contrafreeloading, in which an organism chooses resources (e.g., food) that requires work or effort to obtain when the same resource is offered without effort (see Carder & Berkowitz, 1970; Jensen, 1963; McGowan, Robbins, Alldredge, & Newberry, 2010; Singh, 1970). This phenomenon is well-established and has been observed across many species including rats, mice, pigeons, chickens, crows, cats, gerbils, fish, bears, pigs, and humans (see de Jonge, Tilly, Baars, & Spruijt, 2008; Lindqvist, Schütz, & Jensen, 2002; Mendez, 2002, for reviews), though not consistently in pigs (Young & Lawrence, 2003). It has also been observed across reinforcers such as food, water, sucrose-flavored liquid, and even access to a mirror. Further, this phenomenon appears to occur regardless of explicit training (e.g., Neuringer, 1969). Thus, the robustness of this phenomenon may indicate that in more barren environments, work itself becomes a property of preference, which may disrupt the behavioral allocation that matching would predict when reinforcement is based simply on relative ratio of reinforcement.

A recent study indeed characterized contrafreeloading using the matching relation and showed that previous training with different reinforcement rates can partially explain this preference for work, i.e., experience with one source of reinforcement may enhance behavioral momentum with that source (Podlesnik & Jiminez-Gomez, 2016). The variability observed in contrafreeloading also comes from other parameters as the response requirement for the response-contingent source (e.g., Jensen, 1963; Milella, Amato, Badiani, & Nencini, 2008), the schedule of reinforcement used (cf., Neuringer, 1969; Rachlin & Baum, 1972; Young & Lawrence, 2003), and stimulus changes associated with reinforcement (e.g., de Jonge et al., 2008; Wallace, Osborne, Norborg, & Fantino, 1973). The interested reader might consider Inglis, Forkman, and Lazarus (1997) and Osborne (1977) as reviews for some of these parameters.

These examples of laboratory studies indeed have useful applications to behavior in animal care settings. Contrafreeloading has been used as a form of enrichment in zoos, for example. Grizzlies were given apples and salmon freely or earned by manipulating ice that contained food (McGowan et al., 2010). Although the bears ate more apples and salmon that were given freely, they spent considerable time manipulating the ice containing food, especially compared to ice that was empty of food. The tendency for animals to choose more effortful responses is also exemplified in other studies (Clement, Feltus, Kaiser, & Zentall, 2000). Chimps in a study at a cancer research institute could choose to have their food delivered noncontingently or could be trained to learn a new task that had to be completed before their food was delivered. They consistently chose to learn the new task over the passive delivery of food. The authors concluded that the training was enriching and preferred by the animals.

It may be the case that contrafreeloading only occurs in the context of a barren or captive environment—one with reduced access to reinforcers. Most of the studies on contrafreeloading were conducted in laboratory settings it is unclear whether this phenomenon exists in somewhat more enriched environments like zoos. One study (Coburn & Tarte, 1976), however, examined contrafreeloading comparing rats raised in enriched environments (with a sand pile, various toys, ladders, and tunnels that were rearranged every few days) with those in impoverished environments (standard laboratory cage). Those in the impoverished environment learned to bar press immediately and spent 43–79% of their time bar pressing when given a choice between free and contingent food. Those in the enriched environment made few bar presses and spent less than 1% of the time bar pressing for food. In short, work, i.e., bar pressing, seems to become valuable when few alternatives are available in its environment.

Discussion

The main purpose of this review was to offer a behaviorally based approach to conceptualizing the PWB of captive animals. The approach offered here is grounded in findings from experimental psychology—much of it in the experimental analysis of behavior—and involves two specific considerations. First, we consider the presence of behavior that is not normally found in the wild—stereotypies, inactivity, and displaced aggression—and the conditions under which they occur. Second, PWB was examined in the context of an austere environment, in which an animal displays no problematic behavior, but little operant, or voluntary, behavior. In this case, it is useful to examine whether operant behavior and the ability to vary and influence the environment might have value to the animal.

This review is intended to draw attention to a long and rich experimental literature that can be used to quantify the value of having an enriched environment, which translates into having the opportunity to engage in a range of behavior that changes the environment. This review is also intended to expand discussions of broadening the definition of a healthy environment for a captive animal to include the social environment (e.g., housing with conspecifics vs. solitary), the living environment (e.g., what type of food, how large the cage), and the manipulative environment (the opportunity to engage in operant behavior, i.e., "to work for reinforcers"). Research institutions tend to follow federal guidelines that underscore health for understandable reasons. Biomedical research also requires that conditions be held constant except the variables of interest, and those are varied systematically. Introducing highly variable environments to a cancer, endocrine, or neuroscience study, for example, could increase the cost several-fold while potentially diminishing the experimental power by introducing unplanned variability. This, of course, might undermine the very reason that the animal studies are being conducted.

On the other hand, systematically introducing this variability or enrichment might promote generalization to human health by mimicking the complexity of human environments better. Indeed, current federal regulations suggest that rodents should be housed together in research colonies (Spangenberg, 2007; National Research Council, 1996) and many considerations are outlined, such as enclosure and floor size, as well as the social characteristics of the breed. Our biomedical research enterprise models free-feeding animals (which will become obese) that exist in barren, unstimulating environments and, accordingly, are woefully inadequate to modeling the true human condition. One may wonder whether our animal models do a better job of modeling overweight and sedentary people than the population at large. It is surely worth considering the investment to do better.

In other institutions, like zoos, a varied and enriched environment is more important than in conventional laboratories. These institutions could even serve as “soft laboratories” for examining the importance of allowing for species-specific behavior and creating environments that mimic the wild. For example, some zoos have set up a simulated foraging condition in which animals, such as leopards, search and track their food using acoustic stimuli ( Fàbregas, Guillén-Salazar, & Garcés-Narro, 2012; Markowitz, Aday, & Gavazzi, 1995; Markowitz, 1977, 1973). Others use enrichment items, such as KONG balls and boomer balls, or snacks or food that is hidden or placed in containers that require manipulation to open (Hoy, Murray, & Tribe, 2010). These apparatuses can be used daily and general activity levels increase after the enrichment device is placed in effect (see, e.g., Alligood & Leighty, 2015; Kagan et al., 2015).

It is tempting to consider creating environments in which all food or other reinforcer response-contingent. Animals may exhibit fewer behavioral problems and more activity and interaction with the environment. In fact, the difference between living in a cage and being fed on a schedule versus the same condition plus having daily 30-min sessions in which lever pressing produces food, is revealed in the startling differences seen in brain neurochemistry (Cory-Slechta et al., 2008; Virgolini et al., 2008). The differences are so striking that they raise questions about the validity of studies conducted in animals housed under standard laboratory environments in which animals are allowed to grow obese while housed in a cage that offers few opportunities for meaningful interactions with their environment. This raises a host of economic concerns, however. Furnishing an apparatus that delivers food contingently upon responses (e.g., Markowitz et al., 1995) requires money and time, depending on how intricate the system is. However, less expensive forms of enrichment are available, such as bottles, cardboard boxes, and other inexpensive objects that allow for manipulation (Videan, Fritz, Schwandt, Smith, & Howell, 2005). However, wear and weather can compromise the use, effectiveness, and potency of these enrichment items, so regular inspection and evaluation is required.

Another possibility is the construction of a simple apparatus that delivers food contingent upon responses. The conceptualization of a closed economy used in behavioral economic studies is relevant here. Within closed economies, animals are housed in quarters (usually an operant chamber) in which all of their food is accessed ad libitum but contingent on responses (Collier, Hirsch, & Hamlin, 1972; Hursh & Silberberg, 2008). Again, the cost is higher in these situations. Other less expensive alternatives might include something as simple as a KONG ball, in which all food is placed inside the ball throughout the day, and the organism must interact with it, i.e., roll it or shake it, to produce the food.

In choosing enrichment devices, it is also wise to exercise caution. Certain types of reinforcers in conjunction with other parameters may have severe health consequences to the animal. For example, placing a running wheel into an environment, and delivering food on a fixed basis may induce excessive wheel-running that may lead to activity-induced anorexia or self-starvation (Boakes, 2007; Epling & Pierce, 1992) or severe ulcers in the lining of the stomach (Boakes, 2007; Paré, 1975). This is not only harmful to the animal, but also costly. For a researcher, the replacing of the animal and time lost figures into the total cost.

Consideration of environmental enrichment apparatuses also includes the other costs and benefits associated with the possible impact on the research. The development of U.S. federal regulations was beneficial to scientists to a large degree, because it minimized confounds of health-related problems of the subjects by limiting the biological constraints placed on the animals (Trepanowski, Canale, Marshall, Kabir, & Bloomer, 2011). This may also be true in dealing with PWB. If an animal is tail-biting or self-mutilating in a standard laboratory environment, it may have an impact on the performance of the animal during behavioral experiments and its impact on results from other biomedical areas is unknown, in particular if painful or distracting lesions are apparent. An enrichment device or a variable schedule of feeding may be called for such that the injurious behavior may be reduced.

There are other reasons that an informed decision about housing conditions are warranted. It would be counterproductive if such decisions worked against the experimental goal. For example, a closed economy (see Collier et al., 1972; Hursch & Silberberg, 2008) may mask results if it is incorporated into the experimental question, in particular if the researcher is a behavioral economist, because it changes the value, i.e., demand, for food during the experimental session. Enrichment may also modulate effects that a toxicant may have on brain or behavioral function (Faherty et al., 2005; Guilarte, Toscano, McGlothan, Weave, 2003; Tyler & Allan, 2013), increasing the chances of a type II statistical error.

In short, in evaluating whether operant behavior should be part of an enriched environment, the following might be considered: economic cost and funding, the degree to which problematic behavior of the animal exists and whether that behavior is harmful to the current research question, the degree to which adding an alternative may harm the animal or the research objective, the degree to which the animal will respond to the apparatus, and of course, one’s own personal ethics. The current standards for animal welfare in the United States place a heavy emphasis on health and ethological concerns. Less concern is given to opportunities for operant behavior. An environment that facilitates operant behavior is the type of setting that provides the animal with the variability and control over the surroundings that facilitate thriving and this will lead to more valid models of the human condition. The implications this has on economic resources and science itself are complex, but are certainly worthy of consideration and additional research.