A Commentary on Critchfield and Reed: The Fuzzy Concept of Applied Behavior Analysis Research

There are few opportunities that can entice me to leave my blissful life of retirement, even temporarily, but commenting on the paper by Critchfield and Reed (2017) is, happily, one of them. One of the reasons Donald Hantula, editor of The Behavior Analyst, asked me to comment on the paper is because of the connection I had to the University of Kansas in its early days as a center for Applied Behavior Analysis (ABA) research. To quickly summarize my involvement with the University of Kansas: after receiving my doctorate in Disabilities from Florida State University in a 1970, I did a postdoctoral fellowship under the guidance of the pioneer of ABA research in Regular Education, the iconic R. Vance Hall. Vance was the director of the Juniper Gardens Childrens’ Project, in urban Kansas City, Kansas. During my fellowship I interacted with 8 to 10 doctoral students whom Vance was advising. I also made a few trips to Lawrence, Kansas, where the University of Kansas and most of the ABA faculty and graduate students were located. In Lawrence I interacted with several other graduate students, but only minimally.

It was assumed, correctly, that all graduate students at Kansas had read and understood BWR.1 There was little discussion as to whether BWR got it right; the paper was generally treated as gospel. Instead, with the research template BWR generated, the Juniper Gardens’ graduate students went into the public schools, ran their studies, collected data daily, did periodic interobserver reliability checks, and ran experimental manipulations that verified the superiority of intervention conditions over baseline conditions.

To understand the impact ABA and BWR had on our behavior, it is necessary to be aware of two concurrent events. One is that the spirit of the Sixties was very much alive in 1970. Young people approached the world’s problems with remarkable energy and optimism. We felt that we could change the established order and create a better society. The other event is that the University of Kansas had, in a short period of time, assembled a remarkable group of applied behavior analysts, including all three authors of BWR. So idealism was in place, as was the mechanism to realize our goals, or so we thought. Some, such as I, later left Juniper Gardens and the University of Kansas and took up academic careers. For me, it was 2 years at the University of Connecticut and 40 years at Temple University.

I believe that most of us who had a connection with the University of Kansas in its early ABA days were disappointed that we did not have more impact on society’s problems. I certainly am. Critchfield and Reed’s (2017) article gives us some idea as to why this may be the case - that strict adherence to the research methodology specified in BWR has prevented ABA researchers from investigating some of society’s most important questions.

Before getting into the specifics of the challenges presented by BWR, anyone who wishes to scientifically study society’s problems first has to make a case that science is even relevant, given the strongly held positions that many individuals and groups hold on almost any issue of importance. For applied behavior analysts, there was a second problem. Most respected research of the time involved large groups of subjects, discrete (i.e., occasional) measurement of behavior, and a comparison of randomly assigned groups, each experiencing a different experimental condition. The data were usually analyzed with inferential statistics. Then came BWR and its effort to create a new manner of doing scientific research. Codifying much of Skinner’s (1938, 1953) earlier work and Sidman’s (1960) research tactics, BWR made the case for a new type of research methodology. Now it was legitimate for a published study to have as few as one subject; it was preferable to measure behavior frequently, usually daily; the reversal and multiple – baseline designs replaced the group -comparison design; and the value of an intervention was evaluated according to whether or not a problem had been solved, not according to a statistical test.

It is not surprising that BWR was rejected by researchers who were using traditional scientific procedures. Few people relish the idea of having their careers delegitimized and referring to the group - comparison design as a true experiment (Campbell, 1969; Campbell & Stanley, 1963) doesn’t leave much room for competing practices. But Critchfield and Reed (2017) make the case that BWR has also caused problems for scholars who wished to do ABA research.

The Critchfield and Reed (2017) paper should not be seen as a refutation or denigration of BWR. Quite the opposite. The authors give BWR full credit for affording applied behavior analysts a research methodology by which to study socially - important behaviors. But key to their paper is that BWR is a template for studying behavior but not the only template, arguing that there is not, and cannot be, a single one-size methodological definition of ABA research methodology. ABA broke with some of the characteristics of laboratory - based operant - conditioning research, but also maintained some of its uniqueness. The farther a behavior was from being a free operant, and the less likely a behavior was to revert to baseline levels when an intervention was removed, the less efficacious frequency measures and reversal designs were for studies on human behavior. A frequency record was a good measure of how many times a child tapped on her desk, but not for how well she could summarize a story she had just read. A youngster who learned to walk with the help of an edible reinforcement procedure was likely to continue walking when the intervention was formally terminated. Critchfield and Reed propose that in order to become more socially relevant, applied behavior analysts do not need to adhere to all seven of the research standards called for in BWR. Instead they recommend a relaxation of some of the criteria, when appropriate, allowing ABA researchers to investigate a wider range of meaningful questions.

BWR objected to indirect measures of behavior such as verbal reports. Critchfield and Reed (2017), however, make a convincing case that this restriction prevents applied behavior analysts from studying important behaviors, such as engaging in drug abuse, incurring injuries, and failing to comply with medical recommendations–behaviors that may not be directly observable. In these cases, Critchfield and Reed promote the use of indirect measures. In some cases a behavior may presently be observable indirectly, but with advances in technology becomes directly observable. For example, at one time the best measure of an insomniac’s sleep duration may have been a self-report. With the advances in technology that we are presently experiencing, however, automatic and direct measures are feasible.

My recommendation to ABA researchers on this issue is not to consider the measurement procedure or the research design as the most critical parts of a study; instead, they should regard the research question as the most important aspect of any study. Next, researchers should use the most scientific measurement procedures and the best research designs that are feasible. If the research question can be examined according to the operations recommended in BWR, behavioral scientists should use them. They are very precise experimental operations. But often this will not be the case. Should a researcher then abandon the research question and look for research questions that fit BWR? Absolutely not! My proposal is that researchers use the best measurement procedures and research designs that are available, even if they are not ideal. If a direct measure is not available, investigators should use the best indirect measure that is attainable. If one of the research designs recommended by BWR is not feasible, and a group - comparison design is possible, researchers should use it, without hesitation. This is not a compromise with science; it is a reasonable response to reality. If the editors of ABA journals reject articles because they do not strictly conform to the requirements of BWR, another journal will likely publish the study, and ABA will be less relevant. Critchfield and Reed (2017) point out that the Lovaas (1987) study does not meet BWR requirements. But who would doubt that the Lovaas study is the most important in history for demonstrating the effectiveness of ABA techniques for children with autism? Perhaps bad research is worse than no research at all; but an approximation of ideal research is still good research, and a frequent alternative to using ideal research practices is to do no experimentation at all and to relegate the public to the testimony of advocates and stakeholders.

The recommendations of Critchfield and Reed (2017) are reasonable. Experimental procedures are a means to an end – tools for discovering important relationships. ABA researchers must revise their research procedures, when the situation calls for it. Applied behavior analysts have shown a capacity for changing their research practices, as the field has progressed. In addition to using the standards promoted in BWR, most ABA research now includes measures of social validity (Wolf, 1978) and treatment integrity (Cooper, Heron, & Heward, 2007, pp. 235–237), and often ABA articles report effect sizes (Cohen, 1988). A cumulative field, such as ABA, must evolve in order to remain relevant.

An editor once asked me to review a book, but on the condition that I find something about the book to criticize. Although I do not agree with the editor’s philosophy, I do have one quibble with the Critchfield and Reed article. The title and text of the article use the word, “fuzzy,” to describe the new concept of ABA research. The authors offer a reasoned, technical explanation for their choice of the word “fuzzy,” but I still believe it detracts from the seriousness of their article. People who only see the title of the article may regard the work as somewhat frivolous, which it clearly is not. My preference would be that they refer to the science of ABA as an “evolving concept.”

Inevitably, many readers will wonder how Donald Bear, Montrose Wolf, and Todd Risley would react to the Critchfield and Reed (2017) article were they still alive. Likely, pretty favorably. Critchfield and Reed cite a statement from Risley that he and Wolf intended to write an article that was heuristic and not definitive. Also, despite Wolf’s warnings against conducting laboratory analogues, the Journal of Applied Behavior Analysis published an article on self injury, using a monkey as a subject (Schaefer, 1970). In addition, Wolf was the first author of study using a control group (Wolf, Giles, & Hall, 1968) and made a case for including measures of the social validity of ABA interventions (Wolf, 1978) as a way for ABA to “find its heart.” Finally, at the end of his career, Risley was an outspoken proponent of Positive Behavior Interventions and Supports (Horner & Sugai, 2015; Tincani, 2011).

Compliance with Ethical Standards

Conflict of Interest

The author has no conflict of interest.