Variability As a Subject Matter in a Science of Behavior: Reply to Commentaries

In my article I claimed that studies of operant variability that use a lag n or threshold procedure and measure the obtained variability through the change in U value fail to provide direct evidence that variability is an operant dimension of behavior. To do so, I adopted Catania's (1973) concept of the operant, which takes the increase in the overlap between an R distribution and an S distribution as the behavioral process that demonstrates an operant relation. This increase in overlap can be measured only if the reinforcement criterion and the measure of the effect of differential reinforcement are both defined on the same response property. Because the cited studies of operant variability defined their reinforcement criterion on a sequence property P and measured the effect of differential reinforcement on another sequence property P′, their results cannot provide direct evidence that an operant relation was established.

The commentators raised general points concerning (a) Catania's (1973) concept of operant; (b) the adequacy of applying Catania's concept to behavioral variability as an operant class; (c) the concept of differentiation and the role it plays in operant conditioning; (d) the possibility that more basic behavioral processes underlie the acquisition of behavioral variability in studies that measure variability in sequences of responses; and (e) the procedures and data of specific experiments designed to investigate operant variability.

I sincerely thank the commentators for their useful remarks on my article. The points they raised led me to see some aspects of my position that I had not yet considered and gave me a chance to clarify some aspects of my thesis. In what follows I will try to address the questions that the commentators have raised.

Allen Neuringer

Random human behavior

Neuringer (2012) noted that his findings (Neuringer, 1986) have shown that random-like behavior can be reinforced. First, I would like to emphasize that in my article I discussed the claim that variability is an operant property of behavior like force, duration, location, topography and so on. Neuringer's (1986) results clearly showed that statistical feedback, in conjunction with instructions and descriptions provided by the experimenter, can lead human subjects to produce sequences of numbers statistically indistinguishable from sequences produced by a computer-based random number generator. This feedback was based on distributions of scores of some statistical descriptors obtained for series of numbers that the computer generated.

I would like to point out two aspects of Neuringer's (1986) procedure: (a) The feedback consisted of a complex programmed consequence (a table that related statistical descriptors and classes based on distributions of the computer-generated statistical descriptor scores); and (b) during the feedback condition, subjects were continually instructed about the task. “All questions were answered truthfully as to how the various descriptors were calculated. Furthermore, the experimenter continually suggested ways to improve performance” (p. 65). Although instructions were minimized in Experiment 2, the subjects were still informed about how each descriptor had been computed and how different performances might affect it. Therefore, the effect of the feedback (itself a complex programmed consequence) was presumably modulated by instructions provided by the experimenter. As Neuringer (2012) admitted, “The feedback at the end of each trial was presumed to serve as a conditioned reinforcer” (p. 229). Why was the feedback presumed to serve as a reinforcer if its effect on the randomness of the subjects' sequences was quite clear? Perhaps the function of feedback can only be presumed (and not affirmed) because it consisted of a programmed consequence whose effect on behavior was mediated by instructions and descriptions provided by the experimenter.

We could then ask: Did Neuringer's (1986) feedback have the same function as the food pellet that is produced by lever pressing in a rat? It seems that qualitatively different processes underlie each of these conditioning examples. Thus, Neuringer's results can hardly support the claim that randomness is an operant property of behavior like force, duration, location, topography and so on.

Percentage of variation as variability measure

Neuringer (2012) noted that in many studies variability was measured through changes in percentage of variation (percentage of sequences that meet the contingency requirement). In these cases, the reinforcement criterion and the measure of the effect of differential reinforcement were not dissociated. Furthermore, the results have shown that variability contingencies controlled the changes in percentage of variation as well as the changes in U value. If an experimenter uses a lag n contingency (and therefore defines the reinforcement criterion on the sequence property FPN), I can conclude from the change in percentage of variation that property FPN is an operant property of sequences. Note that under a Lag 4 contingency, for example, percentage of variation equals the percentage of sequences with FP4 equal to zero. It is certainly true that percentage of variation and U value are correlated measures of the lag n contingency effect, but, as I pointed out in my article, this correlation is not perfect.

To illustrate my argument, suppose that an experimenter requires subjects to emit sequences with a minimum cumulative U value, as I suggested in my article. Here the experimenter can also calculate the percentage of sequences whose cumulative U value equals or exceeds the minimum U value. Here also the reinforcement criterion and the measure of the effect of differential reinforcement are not dissociated. But what would it mean to calculate, for example, the percentage of sequences with FP4 equal to 0 under such a contingency? Although the U-value contingency might increase the percent of sequences with FP4 equal to 0, the change in percentage of sequences that meet the U-value contingency is in this case the appropriate measure to demonstrate that cumulative U value is an operant property.

Variable behavior as a generalized operant

According to Neuringer (2012), in so-called generalized operants, measures of behavioral processes and reinforcement criteria need not be identical. Imitation is an example. Variable behavior could be another one. I agree that Catania's (1973) concept of the operant does not apply to all cases of operant conditioning, but I think it can perfectly apply to the typical studies of operant variability, because in these studies the experimenter defines a reinforcement criterion based on a quantitative and measureable sequence property (its absolute frequency in the previous n trials or its weighted relative frequency), and measures the effects of differential reinforcement on a quantitative and measureable sequence property (its cumulative U value). Both properties vary in a single dimension, both can define a reinforcement criterion, and both can serve to measure the effect of differential reinforcement. In these studies, therefore, the experimenter can perfectly define reinforcement criterion and measure the effects of differential reinforcement on the same sequence property. In doing so he or she can demonstrate that a behavioral measure based on the sequence property P changes consistently with differential reinforcement because sequences displaying some particular realizations of the property P produced certain environmental consequences, and not because of any other reason. Only in this case can the experimenter produce direct evidence that P is an operant property of sequences. Catania's (1973) concept requires just this coincidence to demonstrate an operant relation. This coincidence is a necessary condition for such a demonstration. The effective demonstration is achieved only if the measure of the effects of differential reinforcement changes in specific ways during differential reinforcement.

Furthermore, I suggested that contingencies based on the property FPN and contingencies based on cumulative U value could engender performances that are fundamentally distinct from each other (the distinction between local variability and molar variability). Investigation of the specific effect of each contingency constitutes, in my opinion, the heart of an experimental analysis of behavior.

U value as a general measure of variability

By general I meant simply the most common measure adopted in studies of operant variability. I did not mean that the U value can replace or represent other variability measures.

Differential reinforcement of switching behavior

I discussed Machado's (1997) study in my article because it offers a perfect example of a procedure in which the sequence property that defines the reinforcement criterion (number of switches) and the measure of variability (the change in proportion of different sequences) were clearly dissociated. The study is an example of a procedure that is unable to demonstrate that variability is itself an operant property of sequences even if it engenders high levels of variability. Machado's study is particularly useful to illustrate my argument because of its procedure. Its results are not essential to my argument. What is essential in Machado's procedure is that it does not allow one to demonstrate that variability is an operant property of sequences, even if it is, because Machado defined his reinforcement criterion based on a sequence property dissociated from the variability measure he adopted (in other words, reinforcement was not made contingent on sequence variability).

Armando Machado and François Tonneau

Differentiation processes

Machado and Tonneau (2012) noted that differentiation can be seen as (a) synonymous with reinforcement, taken as a behavioral process (thus the claim that differentiation is the process that demonstrates operant relation is trivial); or (b) a component of operant reinforcement (which also includes induction and extinction). In the second case, the operant relation can be demonstrated without measuring the differentiation process.

As I pointed out in my response to Neuringer, the coincidence that Catania's (1973) concept of operant requires to demonstrate an operant relation is perfectly achievable in typical variability studies. Whether we call the behavioral process measured differentiation or not does not change my argument.

Negative frequency-dependent selection

Machado and Tonneau (2012) pointed out that Machado (1989, 1992, 1993, 1997) has identified a common thread that permeates all procedures used to differentially reinforce variability: They all promote negative frequency-dependent selection. According to Machado and Tonneau (2012), the alternative account that Machado sketched attempts “to fill the causal gap between reinforcement (S) and response (R) distributions” (p. 253). But it is not clear how this common thread can fill the gap that I pointed out in my article between the property used to define the reinforcement criterion and the property used to measure the effects of differential reinforcement. It seems that filling this gap is a procedural issue (i.e., the gap is filled only if the reinforcement criterion and the measure of the effect of differential reinforcement are both defined on the same sequence property).

Operant variability conditioning as a reducible process

Machado and Tonneau (2012) also lamented the discrepancy between the reinforcement criterion and the measurement of its behavioral effects, but they did so because it prevents one from identifying possible elementary processes that underlie the conditioning of operant variability. I do not exclude the possibility that the conditioning of operant variability (whatever procedure is used to determine it) can be reduced to more elementary behavioral processes. What I hold in my article is that the coincidence between the sequence property that defines the reinforcement criterion and the property used to mesure the differential reinforcement effects is a necessary (not a sufficient) condition to prove that a given sequence property is an operant property.

Per Holth

The alternative explanation

Holth (2012) also mentioned the possibility that variability engendered by a lag n or threshold procedure in sequences of responses is the effect of more elementary behavioral processes. It is possible that programmed consequences in typical variability studies do not actually affect the sequences as a whole, but instead may act on discrete responses that compose the sequences. In this case, the variability obtained as the effect of lag n or threshold contingencies could be explained without assuming that variability itself is an operant property of sequences. I readily admit such a possibility, but this issue is not directly related to the point I made in my article. I think I can clarify the point by commenting on a procedure Holth suggested to investigate operant variability. Holth proposed that an experimental arrangement that requires subjects to emit simple responses on different operanda (instead of different sequences on just two operanda) could allow the experimenter to circumvent the problem of the sequence complexity. Suppose an experimenter uses pigeons to study operant variability and employs a panel with a matrix of 4 × 4 response keys. Suppose also that the experimenter requires the pigeons to emit responses with locations differing from those of the previous fours pecks (a Lag 4 contingency). If the experimenter adopted U value (calculated for an entire session over the relative frequency of each of the 16 locations) as a variability measure, I could still point out the same discrepancy that I have identified in the typical variability studies.

M. Jackson Marr

Catania's (1973) concept of the operant

Marr (2012) called attention to the limits of Catania's concept of the operant. As I mentioned earlier, I agree that this concept does not apply to all cases of operant conditioning. Essentially, Catania states that a specific change in the R function (from the Ri function to the Rf function) is the behavioral process that demonstrates an operant relation. This change is defined relative to an S function. What matters is how the relation between the R and the S function changes from an Ri function to an Rf function. A change in which the overlap between the R function and the S function increases (as a result of differential reinforcement) is the effect that demonstrates an operant relation. Catania suggests a way to measure this change in overlap. This measurement requires that the R function and the S function are both defined based on the same response property. The superposition of an S function (or a feedback function, as Marr suggested) over an R function divides the total area under the curve of the R function into two distinct subareas. Marr suggested measuring the effect of differential reinforcement by calculating the difference (from the Ri function to the Rf function) between the correspondent subareas.

Thus, measuring the effect of differential reinforcement, by Catania's (1973) reasoning or any other that essentially adopts the same framework (like the one proposed by Marr), requires that there is a fixed criterion of reinforcement (represented by an S function or a feedback function). If a reinforcement criterion does not exist or if it does exist but is not fixed, Catania's reasoning cannot apply. One example of this is superstitious behavior. Because there is no reinforcement criterion at all, it is impossible to adopt the measurement that Catania proposed during the acquisition of a superstitious operant. Although the R functions of some response properties change, there is no reinforcement criterion by which the changes in R functions can be evaluated. Another example is an operant that is shaped by percentile schedules of reinforcement (Platt, 1973). Percentile schedules of reinforcement establish a changing reinforcement criterion so that the proportion of criterion responses is held constant. By using the calculus tool Marr suggested, it could be said that percentile schedules of reinforcement shape operant behavior without changing the criterion area under the curve of the R function. Ideally, the difference would be kept equal to zero even though the R function continually changes. Because the reinforcement criterion is also continually adjusting (according to the subject's own recent behavior), the proportion of reinforced responses is held constant.

Machado (1989) used a percentile reinforcement schedule to differentially reinforce variable sequences and found that levels of variability were a function of the contingency requirements. Here Catania's (1973) reasoning could hardly apply, not because of some special feature of variable behavior, but because of a special feature of percentile schedules. Similarly, if a time schedule generated superstitious emission of variable sequences, Catania's reasoning could not apply, not because of some special feature of variable behavior, but because of a special feature of the time schedule (a schedule that does not establish a contingency relation between response and reinforcement). The typical studies of operant variability I have discussed establish a fixed reinforcement criterion on a sequence property.

U value as reinforcement criterion

A computer program could assume that a subject has emitted each of the 16 different sequences exactly once before the subject has actually emitted any sequence (the subject enters the experiment with a cumulative U value equal to 1). After the emission of each subsequent sequence, the program could recalculate the cumulative U value (by taking the 16 virtually emitted sequences with which the subject started plus the actually emitted sequences during the experiment) and provide reinforcement only if the current U value is higher than a predetermined value. This is naturally only one example of a contingency that could be established based on the cumulative U value. Given that the usual U value ranges from 0 to 1, another example could consist of a contingency that makes the probability of reinforcement equal to the current cumulative U value. This is an example of the linear feedback function Marr mentioned. Marr pointed out, however, that even a contingency based on cumulative U value cannot reveal how the final performance is shaped. In fact, in this case there is also the possibility that more fundamental processes could explain variable behavior (if it were produced). This alternative explanation, if confirmed, could invalidate the claim that cumulative U value is an operant sequence property. The advantage here, however, is that the experimenter can obtain direct evidence that cumulative U value is an operant property of sequences (if it is).

Conclusion

Relevance of establishing correspondence between sequence properties

The main point I have made concerns the desirable coincidence between the sequence property that defines a reinforcement criterion and the sequence property used to measure the effects of differential reinforcement in variability studies. From the point of view of behavioral technologies, it is possible that this coincidence is not relevant. For example, suppose a behavior analyst who works with patients needs for some reason to increase the variability of responses one patient emits (irrespective of whether the patient varies his or her responses only over extended periods of time, often repeating the more recent ones, or the patient rarely repeats the more recent responses). If this behavior analyst is told that a lag n contingency efficiently produces such an effect, he or she can expose the patient's responses to a lag n contingency and measure the changes in U value (irrespective of whether U value increased because consequences were made contingent on a certain minimum cumulative U value or because of any other operation). Note, however, that it would also be irrelevant to this behavior analyst whether variability is an operant property of behavior or the increase in variability is an indirect effect of more basic processes, as Holth (2012) and Machado and Tonneau (2012) suggested. I think that it is essential to the behavior analyst who works as a researcher focused on behavior in its own right to determine exactly why a behavioral measure has changed.

Variability as a subject matter in a science of behavior

Variability as an object of study for behavior analysts has not been shown to be a conceptual unit. In studies on induced variability, variability is often taken as synonym of dispersion and is measured by descriptive statistics (Barba, 2006). Studies on operant variability have discussed variability as synonymous with unpredictability or randomness (Neuringer, 2002). In his commentary, Marr (2012) pointed out the problem of determining whether an event (or even a set of events) is random or not. Neuringer (1986) noted that the mathematical meaning of randomness normally involves infinite sequences of events. All the experimenter can do is to try to indirectly access the presence of this property in a finite event sequence through statistical evaluations. But “no matter how many statistical evaluations indicate that a finite sequence is random, there may exist some other tests that shows nonrandomness. There is no conclusive test, or set of tests, to prove the randomness of a finite sequence” (Neuringer, 1986, p. 63). It seems that randomness, as a mathematical abstraction, corresponds to some property that is not directly measurable at all. It can only be indirectly estimated. Has this mathematical abstraction any behavioral meaning? Neuringer (2012) seemed to embrace a multidimensional concept of variability when he affirmed, “It should be noted that variability has many dimensions (there are many different ways for a phenomenon to vary or not) and that significant effects in any one or more measures suffice to show a variability difference across two conditions” (Neuringer, Kornell, & Olufs, 2001, p. 90). From a strictly analytic point of view, there is no reason to treat variability as a behavioral property that can only be indirectly accessed by measuring changes in many different dependent variables. At best those variables are in some degree correlated with each other. An analytic effort should seek to investigate the change in them (and possibly the interactions between them) without assuming that they are different ways of accessing the same behavioral property. It is even conceivable that the terms behavioral variability or behavioral randomness could be abandoned in favor of an approach that deconstructs and investigates each variability measure in its own right.