Editorial: Choice Studies in Transition

Almost half a century ago, in this very journal, Richard Herrnstein (1961) published his seminal paper “Relative and absolute strength of response as a function of frequency of reinforcement.” Herrnstein showed that in the steady state the proportion of responding on a schedule was equal to the proportion of reinforcement obtained on that schedule: the matching law.

Besides introducing to researchers one of the great quantitative regularities in psychology, this influential paper put front and center the topic of choice in behavior analysis. It also set the terms under which choice would be studied for the next 40 years: steady state behavior, and molar variables. In his paper, Herrnstein was concerned with the asymptotic behavior that emerges after an organism has been exposed to the same schedule conditions for a period of several days, not by the way the moment-to-moment interaction between the organism and its environment lead to those asymptotic performances. The contingencies of reinforcement to which the pigeons were exposed were stable, both in the sense that they prevailed over long periods of time and in the sense that, given the feedback function in concurrent variable-interval (VI) schedules, the experienced reinforcement rate ratio was relatively independent of the animal's response distribution.

With few exceptions (e.g., Shimp, 1966; Staddon, Hinson, & Kram, 1981), these features of Herrnstein's paper have dominated the booming field of operant choice behavior that it launched. Part of the reason is technical: before digital technology became commonplace, the only convenient method of recording moment-by-moment changes in behavior was Skinner's cumulative recorder. But the molar approach was also favored because for many years it succeeded. Many new relations were found, many intriguing puzzles solved.

But as technology improved, a more molecular, moment-by-moment level of analysis began to gain traction: Research appeared that looked at the effect of individual reinforcers on choice performance (e.g., Aparicio & Baum, 2009; Davison & Baum, 2000; Corrado, Sugrue, Seung, & Newsome, 2005; Lau & Glimcher, 2005), that tried to characterize behavior in transition before it reached its asymptotic state (i.e., Banna & Newland, 2009; Bailey & Mazur, 1990; Cerutti & Staddon, 2004; Gallistel, Mark, King, & Latham, 2001), or that studied behavior in unstable environments, due either to frequent changes in the contingencies of reinforcement (i.e., Davison & Baum, 2000; Grace & McLean, 2006) or to the schedule's feedback function (i.e., Palya & Allan, 2003).

These are topics that also will be addressed by the seven articles in this Special Section. Baum's paper provides an overview of his previous research with Michael Davison on the effect of individual reinforcers on choice in unstable environments, casting it in a more general framework, which tries to reconcile a molar approach with the study of the dynamics of choice, while Rodewald, Hughes, and Pitts revisit them in their contribution. In their paper, Davison, Marr, and Elliffe explore complex feedback functions where the behavior of the animal has a major impact on the experienced reinforcement rate. Kyonka and Grace's contribution continues the exploration by Randy Grace and his colleagues (e.g., Grace & McLean, 2006) of concurrent chain schedules where the fixed-interval (FI) terminal links vary randomly from session to session. The paper by Christensen and Grace provides a theoretical framework for conceptualizing those results. McDowell and Popa show how molar regularities such as the matching law could emerge from the local action of a selection-by-reinforcement principle. And Staddon, MacPhail, and Padilla provide another example of simple dynamical modeling.

The study of the dynamics of choice is very recent in behavior analysis. Hence, the basic facts are not yet fully established. There is no agreement yet about models. Even the questions that should be addressed by researchers are far from settled. Many laboratories have been following their own line, with little effort to integrate their efforts with the work of others. Perhaps this is to be expected in a still-young field. We hope that this issue of JEAB, by bringing together contributions from different perspectives on the dynamics of operant behavior, will help stimulate more research and encourage a more integrative approach to this important topic.