Disequilibrium as an Alternative to Internal States and Affordance

Killen and Jacobs (2017) propose a new four-term operant contingency, in which an O (physiological/dispositional/motivational state of the organism) is added to the traditional three-term S-R-S^r contingency. This fourth term is added in an attempt to explain changes in responding that may depend on the state of the organism responding for that reinforcer. We propose, instead, that an older model, the disequilibrium model (Timberlake & Farmer-Dougan, 1991), may already account for changes in such changes in responding. Further, the disequilibrium model may also predict the magnitude and direction of changes in responding across changing contexts.

While reinforcement theorists ponder the function of the environment, discriminative stimuli, and the reinforcing event itself on the rate, magnitude, and probability of response, applied researchers struggle with finding a parsimonious explanation of reinforcement theory that is both experimentally and ecologically valid. Given the differing theoretical explanations and daunting terminology used to describe reinforcement, discerning an appropriate reinforcement model for an applied setting has become confusing. In our canine research, as well as previous work with humans, we see behaviorists in applied settings struggle with “simple” concepts such as the traditional four-square contingency, the Premack principle, and schedules of reinforcement. Indeed, a brief review of talks for the Karen Pryor Clicker Expo 2017, the online Association for Professional Dog Trainers discussion group, or even the BcBA Behavior Analyst Facebook group yields discussions on when to “use Premack” versus positive reinforcement; how to distinguish between using operant and classical conditioning; or whether not giving a reinforcer is a punisher. These issues demonstrate the need for clarification and explanations of reinforcement effects. It seems that traditional models remain problematic for consistent and accurate implementation, and newer theoretical concepts remain elusive for applied behaviorists. Certainly, behavioral researchers have sufficient knowledge about when a “reinforcer” is reinforcing: It depends. And that is the problem: Exactly what are the defining variables upon which a reinforcement effect is dependent, and how can reinforcers predictably, reliably, and effectively be implemented?

Skinner’s three-term contingency defines three critically important variables: the discriminative stimulus (S^D) that predicts the contingency, the response (R) itself, and the consequence, which is typically a reinforcer (S^r) and follows the response, which is predicted, by both the discriminative stimulus and the particular response. Killeen and Jacobs (2017) add a variable to account for the organism itself, O. This variable attempts to account for variability in the physiological, motivational, and dispositional states that the acting organism may experience. However, any clarification of operant behavior or improved predictive value is likely only on a theoretical level. Further, whether this added variable improves predictability, reliability, or effectiveness for the applied researcher is questionable. We argue it does not. We do not argue that changes in the state of the acting organism are not critically important nor do we deny that properties of the discriminative stimuli, the response, or the reinforcing event are not also critically relevant. Instead, we argue that disequilibrium and bliss point models may already address changes within the organism.

Disequilibrium and Bliss Point Models

One important aspect of any reinforcement procedure is the ability to predict a priori the direction and magnitude of the reinforcement effect (Farmer-Dougan, 1994, 1998; Timberlake & Farmer-Dougan, 1991). One group of reinforcement models, the behavior economic models, and in particular, the disequilibrium model (Timberlake & Allison, 1974; Timberlake & Farmer-Dougan, 1991; Farmer-Dougan, 1998) allows for the specification of reinforcement effects a priori using brief (free) baseline assessments. The disequilibrium model is an extension of response deprivation models (Timberlake & Allison, 1974) and is based on the concept of molar equilibria, or behavioral equilibria. It assumes that responding during an unconstrained (free) baseline represents an equilibrium state between the rate of the contingent (Oc) activity (the reinforcer), and the rate of the instrumental (Oi) activity. Reinforcement effects are produced when a contingency schedule constrains responding into a state of disequilibrium by pushing the ratio of the constrained instrumental (I) to contingent (C) activity above or below the unconstrained baseline ratio. This model reframes reinforcement/punishment effects as a schedule that constrains free baseline behavior and specifies the level of occurrence, rates, and/or times. Furthermore, the model formally specifies the conditions under which reinforcement and punishment effects may occur (Timberlake & Allison, 1974; Timberlake & Farmer-Dougan, 1991). Reinforcement, resulting in an increase in the instrumental (I) response, occurs when:

$$I/C>\mathrm{Oi}/\mathrm{Oc}$$ 1

Punishment, resulting in a decrease in the instrumental (I) response, occurs when:

$$I/C<\mathrm{Oi}/\mathrm{Oc}$$ 2

Here, I is the required amount of instrumental response specified by the schedule; C is the quantity of the contingent response provided as a reinforcer/punisher; and Oi and Oc are the free baseline rates of the instrumental and contingent responses, respectively.

An example helps to clarify the equations: A dog might offer a sit only once (Oi), but eat 100 treats (Oc) during an unconstrained baseline session. The baseline equilibrium rate of Oi/Oc responding would thus be 1/100. If the trainer imposes a schedule such that the dog earns five treats (C) each time it sat, this would impose disequilibrium by forcing the ratio of I/C responses to 1/5. According to the model, the dog should respond in such a way as to minimize the disequilibrium or deviation from baseline for the contingent response. Thus, the dog will sit more often in order to approach the rate of treats obtained during free baseline, which by definition is a reinforcement effect. A parallel example can be used for punishment effects: Suppose the dog jumps up on the owner 10 times (Oi), but offers sits only once (Oc) during unconstrained baseline. If the dog is required to sit each time it jumps up, I/C becomes 1/1, which is lower than the baseline rate of 10/1. The number of times the dog jumps up is likely to decrease to maintain the rate of the contingent sit response, resulting in a punishment effect for jumping.

Further predictability regarding the size of the reinforcement effect may be obtained using the minimum-distance bliss point model (Allison, 1983; Hanson & Timberlake, 1983; Farmer-Dougan, 1998). According to this model, there is a bitonic relationship between the rate of reinforcement imposed by a schedule and the strength of the reinforcement effect: The response rate will first increase and then decrease as the reinforcement rate increases. The bliss point model, then, suggests that schedule constraints that produce very high or very low rates of disruption are less effective than moderate levels of disruption. Using a simple fixed ratio (FR) schedule as an example, the rate of response may be predicted by the equation:

$$R=\frac{\mathrm{Oi}+k\left(\mathrm{Oc}\right)}{k^2+1}$$ 3

Here, R is the predicted rate of response, Oi is the rate of instrumental response during unconstrained baseline, Oc is the rate of the contingent response (reinforcement consumption) during unconstrained baseline, and k is the number of units of reinforcement produced per response on that schedule. Mathematically, k is the inverse of the response-to-reinforcer requirement (e.g., on a fixed ratio 5 schedule, k = 1/5 or 0.2). If Eq. 3 is plotted as a function of varying reinforcement schedule values (k), the maximum value of R occurs at an intermediate value of k. That is, the maximum rate of the instrumental responding should occur at moderate reinforcement rates: When equilibrium is moderately disrupted, the largest reinforcement effect is produced. This is shown in Fig. 1.

Fig. 1.

Predicted response rates using the minimum bliss point model

The Disequilibrium Model in Practice

The disequilibrium approach offers a theoretical explanation of reinforcement effects and a means of applying this explanation in real world settings (Farmer-Dougan, 1994, 1998; Timberlake, 1991). The method is relatively easy to use, predictive, and adaptable across settings, organisms, and responses. Further, it avoids confusing theoretical concepts and terms. Data from several projects from our laboratory show that disequilibrium approaches may be a more parsimonious way of framing reinforcement effects, particularly for those who are not theorists, but simply trying to use reinforcement.

Farmer-Dougan (1998) examined reinforcement effects produced by changes in the probability of disruption of self-initiated reaching for toy items with preschoolers. Preschoolers were reinforced for identifying alphabet letters using incidental teaching, a behavioral intervention used to increase social and verbal behaviors in children and adults with disabilities (Farmer-Dougan, 1994; Haring, Neetz, Lovinger, Peck & Semmel, 1987; Hart & Risley, 1975, 1980; McGee, Krantz, Mason & McClannahan, 1983; McGee, Krantz & McClannahan, 1985; McGee, Morrier & Daly, 1999; Neely, Rispoli, Gerow & Hong, 2016). The probability of disruption of an initiation towards a toy was varied, and changes in correct letter identification responses were recorded. The data were plotted using Eq. 3. Results showed that there was an optimal level for disrupting ongoing behavior (approximately 50 to 75% of interruptions), above or below which reinforcement effects greatly suffered.

A second study examined the effects of a modified peer-delivered incidental teaching procedure (Farmer-Dougan, 1994). Incidental teaching was used to increase appropriate requesting of food items during lunch preparation with adults with developmental disabilities. Three pairs of group-home residents participated in the investigation. Initially, the more verbal individual in each pair was taught to prompt their partner to ask for food items during lunch preparation. A significant increase in appropriate requesting was found for these sessions. A significant increase in appropriate requesting to staff and peers during the evening meal was also found, demonstrating generalization. Further, these increases maintained when the teaching program was withdrawn.

Why was incidental teaching so successful? Incidental teaching techniques focus on accurately identifying a reinforcer, increasing generalization, and maintaining the operant response with “naturalistic teaching” and capturing of a “teaching moment” (Hart & Risley, 1968, 1974, 1980). For example, as the child initiates towards a reinforcer item (e.g., a toy) or activity, the teacher immediately imposes a contingency such that access to the item or activity is thwarted until the child emits the required response. The naturally occurring ongoing (free baseline) behavior is momentarily disrupted contingent on an increase in the instrumental response. Incidental teaching is successful because it imposes a momentary disequilibrium in the ongoing stream of behavior (Timberlake & Farmer-Dougan, 1991). Minimum-distance bliss point models further predict that reinforcement schedules that produce moderate disruption of baseline equilibrium should produce the strongest reinforcement effects. Conversely, schedule constraints that produce very high or very low rates of disruption are predicted to be less effective than moderate levels of disruption (Allison, 1983; Farmer-Dougan, 1998).

But can the disequilibrium model identify “dispositional” differences? Most recently, Farmer-Dougan et al. (2016) used a disequilibrium approach to examine how prior experience with training affected rate of responding for food reward. Six dogs highly experienced with operant conditioning and nine inexperienced dogs were assessed for approach responses during a baseline condition. Dogs were then exposed to several conditions in which they were required to initiate towards a trainer. The probability of reinforcement for initiations was varied across conditions. During baseline trials, dogs received a treat when they approached an automated feeder or a human trainer sitting in a chair approximately 1.5 m from the feeder. The number of times the dog approached and touched the feeder or trainer with its snout or paw was counted across the 5-min baseline session (Oi/Oc). During contingent reinforcement sessions, the same trainer sat in a chair in the middle of the experimental room and delivered treats contingent on an initiation across five different 5-min schedule conditions ranging from 0 (none) to 100% of all approaches.

The number of approaches to the feeder or the trainer during non-contingent baseline (Oi/Oc) and approaches to the trainer during the five contingent schedule conditions was obtained and plotted as a function of reinforcer probability. As illustrated in Fig. 2, experienced dogs showed the pronounced bitonic reinforcement effect predicted by the bliss point model. Inexperienced dogs showed a much flatter function. Inexperienced dogs emitted fewer initiations overall, with experienced dogs emitting significantly higher initiations at the 25 and 50% conditions, consistent with the bliss point model prediction.

Fig. 2.

Responses per minute across various approach reinforcement schedules for both experienced and inexperienced dogs

Why would inexperienced dogs emit fewer approach/initiation responses? It could not have been the food treat; these dogs readily took these treats from the feeder during baseline. Examination of the baseline data for the two groups yields important clues: Inexperienced dogs took treats from the feeder at a much higher rate than they took treats from the trainer. For experienced dogs, treat taking was much lower rate. This suggests that the contingency may have imposed differing disruptions in baseline responding for inexperienced versus experienced dogs. More specifically, there may have been two competing contingencies for the inexperienced dogs: While initiations to the trainer (I) was reinforced by food (C), food taking (I) may have been punished by the required approach to the trainer (C). Indeed, inexperienced dogs’ baseline (Oi) rate of approaches to the feeder was significantly higher than that of the experienced dogs. Thus, the resulting combined contingency imposed a different level of disruption for the inexperienced dogs when compared to the experienced dogs.

Using a disequilibrium approach, our data yield an intriguing and very important finding: An identical contingency had very different effects for the two groups of dogs, depending on their learning history. Dogs familiar with working with a trainer responded as if the contingency was highly reinforcing, while dogs unfamiliar with working with a trainer responded to the contingency as if it were both punishing and reinforcing. Dogs with different reinforcement histories respond very differently to identical reinforcement condition. Use of the disequilibrium model allowed us to determine how differences in training may affect the effectiveness of a reinforcer for inexperienced and experienced dogs.

Affordances and Reward

Killeen and Jacobs introduce the concept of affordance, a term borrowed from J.J. Gibson’s ecological approach to perception (Gibson, 2014). Certainly, examining reinforcement contingencies from an affordance viewpoint is intriguing. However, we disagree with the assessment that affordances are “determined by the O that is operative.” We believe that this interpretation is due to a misunderstanding of the affordance concept.

According to Gibson (2014), the perception of affordances is supported by the detection of lawfully structured stimulation patterns that reflect the fit between the action capabilities of the organism and environmental properties. Affordance may be defined as the opportunity for a given action within the environment. However, and importantly for this paper, according to Gibson, “the affordances of the environment are what it offers to the animal, what it provides or furnishes, either for good or ill.” (Gibson, 2014). Gibson, then, defines affordances as being independent of the organism’s ability to perceive the affordance, but are dependent on the physical capabilities of the organism and environmental properties. For example, a chair may afford the act of sitting for an adult, but would not afford the act of sitting for an infant. Similarly, the chair retains the affordance for sitting whether it is upright or on its side and would afford sitting whether the adult chooses to sit or not. The ability for the chair to afford sitting remains regardless of the physical position of the chair, or the dispositional states of the organism. Perhaps, Gibson states it most clearly:

The affordance of something does not change as the need of the observer changes. The observer may or may not perceive or attend to the affordance, according to his needs, but the affordance, being invariant, is always there to be perceived. An affordance is not bestowed upon an object by a need of an observer and his act of perceiving it. (Gibson, 2014, p. 139)

The basic operant laboratory experiment may be interpreted in terms of affordance. There is the affordance for the pressing the lever, the affordance for consuming the reinforcer, and affordances for a variety of other responses (pecking the wall, picking up the shavings in the bottom of the operant chamber, etc.). One could also argue there is an affordance for the contingency. But these affordances exist whether the organism perceives them or not. That is, the response lever and the food pellet retain the affordance for lever pressing and eating, regardless of whether the rat presses the lever or eats the food pellet.

Norman (1999) addressed how dispositional states and past experiences of the organism may affect affordances. He suggests that factors such as reinforcement history or dispositional states do not alter the affordance itself, but alter the perception of the affordance. Thus, a response lever retains the affordance for pressing, but a rat with a history of being shocked for pressing a similar lever may not perceive the affordance in the same way as a rat that has been reinforced with food pellets for pressing the same lever. This, in turn, alters each rat’s perceived action capability. Note, here, that the opportunity to press the lever does not change, but the perceived affordance differs. Affordances, then, exist relative to the action capabilities of the organism, exist independent of the organism’s ability to perceive it, and do not change with dispositional states of the organism. What changes is the ability of the organism to perceive the affordance, and thus the perceived action capability.

Perceived action capability refers to how an organism perceives their environment in terms of their ability to act upon that environment. Research demonstrates that choices about when and how to transition between two different modes of a given behavior are determined by the task-specific fit between action capabilities and environmental properties (Wagman, Langley & Farmer-Dougan, 2017). Human subjects reliably shift from reaching with their arm only to reaching with their arm-plus-torso as objects are placed farther away (Carello, Grosofsky, Reichel, Solomon, & Turvey, 1989; Kirsch, Herbort, Butz & Kunde, 2012; Kirsch & Kunde, 2013; Osiurak, Moragado, & Palluel-Germain, 2008) or alter their stance of differing degrees of a sloped walkway when wearing a weighted backpack (Malek & Wagman, 2008; van der Hoort & Ehrsson, 2014; van der Hoort, Guterstam & Ehrsson, 2011). The method of reaching or preferred stance on a slope differs depending on the environmental changes (distance or steepness or weight of backpack). Similarly, in operant situations, organisms may shift from one response to another depending on the given affordances within the environment: The reinforcement schedule may offer more opportunity for pressing on the left lever rather than the right.

Recently our laboratory investigated how a changing physical environment required a shift between two modes of behaviors by examining the perception of reaching affordances when dogs reached to obtain food reinforcers (Wagman, Langley & Farmer-Dougan, 2017). Nineteen dogs (Canis familiaris) were briefly exposed to a baseline condition in which they were allowed to eat small pieces of hot dog from a small cup attached to a wall-mounted apparatus. The small cup was then gradually raised, so that the dogs had to reach with the head only to gain access to the hot dog reinforcer. As the cup was raised, the behavioral response of the dogs changed: Dogs moved from reaching with the head only to a rearing behavior, in which the dog reared on two hind legs and either held their front paws in the air or placed their front legs on the wall to steady themselves. Post hoc analyses divided the dogs into short and tall groups, based on American Kennel Club categories, to determine if physical size affected the results. The rearing boundary, or height at which the dogs transitioned from reaching with the head only to rearing, was compared across the two groups. There was a significant difference in rearing boundary, with rearing occurring at taller heights for tall than short dogs. Importantly, there was not a significant difference between the rearing ratio (the rearing boundary divided by the shoulder-to-floor height) for the two groups.

Note that in this experiment, the reinforcement contingency does not change. Dogs had to reach to gain access to the hot dog. The absolute value of the hot dog did not change; it was still a ¼ diameter of a ½-inch piece of hot dog. The discriminative stimulus of the cup was identical; the wall and food-treat apparatus did not change. The height of the reinforcer (an environmental property) and the corresponding behavior (perceived action capability) were changed. Indeed, the topography of the response changed significantly and predictably: Dogs moved from a head-only reach to rearing as the hot dog was raised, and the rearing ratio was remarkably constant across the 19 dogs, regardless of size. Further, the dogs gave other behavioral clues as the response neared the rearing boundary: Dogs appeared to pause more at higher heights and showed more foot movement, ears shifted back, lip licking, body shakes, and vocalizations just prior to rearing. These indicate potential changes “within” the organism, or Killeen and Jacobs’s (2017) dispositional/physiological changes. Traditional reinforcement models measuring response rate would not have detected these behavioral changes. Examining changes in affordance allowed examination of the changes in behavioral topography: The potential for reaching with the head-only decreased while the potential for rearing became more likely as hot dog height increased.

So, why might the use of the affordance concept be a potential tool for behaviorists? First, it puts the focus back on the environment-organism interaction, and away from the reinforcer, itself. Too much effort has been placed on the “magic reinforcer.” There is a temptation to focus on the properties of the reinforcer itself, rather than the properties of the contingency between the response and the reinforcing event. Further, the ecological approach does not require the introduction of an organismal variable, such as “O”. Rather, the emphasis is on the affordances within a given environment, the perception of such affordances, and the perceived action to be taken by the organism. A reinforcement model that is consistent with the ecological approach, then, would be a model that allows for evaluating behavior within changing environments, assessing changes in affordance, the perception of that affordance, and the perceived action to be taken by the organism within that environment. We argue that the disequilibrium model does just that. There is no need to add a variable to account for physiological, motivational, or dispositional factors nor is there emphasis on the reinforcer or punisher. The disequilibrium model examines changes in affordances, perceived affordances of the animal, and the resulting perceived action capability.

Take for example the raising the height of the hotdog in our study. Raising the hot dog changed the affordance for reaching for the hot dog: Different responses were afforded due to the changes between baseline and contingency. Simply examining how the imposed contingency of hot dog height altered reaching/rearing compared to baseline allowed us to evaluate reinforcement effects: When the hot dog was below a ratio of 1.54, reaching with the head only was reinforced; when the contingency imposed a reaching ratio of greater than 1.54, rearing behaviors were reinforced. That is, which response was emitted was dependent on changes in the ratio of contingent I/C compared to baseline (Oi/Oc). No variable O was required. The disequilibrium model explanation emphasizes what the organism perceives as the affordance (possible behavior) and focuses on changes in responding (perceived action capability) as affordances are altered within a contingency.

Last Thoughts

In conclusion, if our goal is to communicate a clear and parsimonious model of reinforcement which accurately, reliably, and effectively predicts reinforcement effects across a variety of organisms, responses, and contexts, then the disequilibrium model may be an appropriate model. This model does not preclude experimental investigations that have formed the basis of our experimental literature. There is no doubt that investigating the complex variables that contribute to responding leads to better theoretical understanding, and in turn, better applications of reinforcement. Rather, the disequilibrium model returns the focus to the instrumental response as it occurs within a contextual environment. It allows for dispositional or physiological changes to occur within the organism between baseline and the contingency. Finally, the disequilibrium model may be easily and successfully implemented in applied settings. It avoids and eliminates confusing concepts and terms and helps the behaviorist focus on the response rather than on “what is a” or “what kind of a” reinforcer. The model is consistent with viewing reinforcement in the context of affordance of behavior, and we argue that viewing behavior from an ecological viewpoint may lead to a new line of research that will further refocus emphasis on the instrumental response. We do believe that such an approach will lead to a better understanding of reinforcement and a more parsimonious model for use in applied settings.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethics Approval

This paper describes both past research that was published elsewhere and animal research conducted at Illinois State University.