The Emperor Just Might Be Wearing Pants

Abstract

This is a commentary in response to Dixon et al.’s (Behavior Analysis and Practice, 8(1), 7–15, 2015) article entitled, “Research rankings of behavior analytic graduate training programs and their faculty” in Behavior Analysis in Practice. The severe restriction of range for the metric used to identify faculty productivity and knowledge of research calls the implications drawn from the data into question. Suggestions on how to broaden the metric are made along with implications for doing so. This is an important topic, and many people will need to contribute to a robust conversation about our graduate training programs given the exponential growth we have faced in recent decades.

In their recent article in Behavior Analysis and Practice, Dixon, Reed, Smith, and Belisle (2015) initiated a conversation on analyzing high-quality applied behavior analysis (ABA) graduate training programs. I applaud their bravery and sincere effort to broach a significant but potentially political issue. Like the authors, I believe identifying, refining, and using the correct metrics against which to measure our graduate training programs are both important and challenging. Without question, an evaluation of the scholarly work produced by faculty teaching in ABA programs is worthwhile, both for the benefit of prospective students and for our field. Scholarly productivity should, in fact, be one of the dimensions on which our faculty and training programs are judged.

The method for determining scholarly contributions made by ABA faculty is where my view deviates from Dixon and colleagues (2015). Certainly, measuring publications in leading ABA journals is valuable. Yet drawing conclusions based on an incomplete view of scholarly work conducted by faculty is akin to an overselectivity problem in a child with autism spectrum disorder. Indeed, suggesting faculty knowledge of research is weak if they are not publishing extensively in the identified ABA journals seems premature and potentially inaccurate. I would argue it is as accurate as concluding that the emperor has no clothes when you can only see him from the waist up—perhaps he is wearing pants, and saying he has no clothes might have negative unintended consequences.

A more accurate metric for faculty research activity should be much broader. Many researchers were taught that ABA would have its greatest impact when we successfully communicate with professionals who represent different, but related fields of study. That is, our collective scope of influence would expand dramatically as we published outside our primary journals. I believe this is particularly critical for the field of ABA because all too many teachers, psychologists, counselors, and other helping professionals believe they not only fully grasp the tenets of ABA, but that our technology is so simple it does not require careful examination. That is, our work is ignored unless it is placed right in front of them.

Brian Iwata, the top publisher in the ABA journals reviewed by Dixon and colleagues (2015), serves as a great example of having a truly impressive scope of influence both within and outside of ABA. I recently had the great pleasure of nominating Iwata for the APA Gold Medal Award for Lifetime Achievement in the Application of Psychology. It should come as no great surprise that Iwata’s substantial body of work helped earn him the award. It is noteworthy that Iwata has published in more than 25 different journals in his career, including the American Journal of Medical Genetics, Child and Adolescent Mental Health Care, Journal of Behavior Therapy and Experimental Psychiatry, and Psychological Services. His prolific work published in ABA journals has influenced generations of behavior analysts, but his ability to communicate effectively with a very broad audience increases the likelihood ABA methodology will be accepted and adopted in the interdisciplinary settings in which most clients are served. To assume the impact of publications in non-ABA journals is less valuable than published work in ABA journals seems shortsighted.

An examination of the impact factor of our own journals also underscores the importance of ABA researchers publishing in the journals spearheaded by other disciplines. The impact factor is an indicator of the average number of citations for articles that have recently been published in a journal. The physical sciences have extraordinarily high impact factors. For example, the New England Journal of Medicine has an impact factor of 51.658, the Reviews of Modern Physics has an impact factor of 44.982, and The Lancet has an impact factor of 39.06. The social sciences have much smaller impact factors with the American Psychologist at 6.87, the American Journal of Sociology at 3.476, and the Journal of Economic Literature at 6.919. A number of ABA journals reviewed by Dixon and colleagues (2015) do not calculate impact factors. Of those that do, the Journal of the Experimental Analysis of Behavior has an impact factor of 1.48, the Journal of Applied Behavior Analysis has an impact factor of 1.19, and The Psychological Record has an impact factor of 0.652. The scholarly work that is published in our top ABA journals is referenced infrequently compared to other social sciences. Thus, we need to expand our scope of influence by publishing not only in our leading journals but also in the top journals of other disciplines. Because Dixon and colleagues (2015) identified ABA faculty as a part of their methodology, an extension of their analysis to other journals (excluding articles that were unrelated to ABA) could easily have been conducted. I hope this is considered in their future examinations of scholarly activity by ABA faculty.

Lastly, even if the authors choose to maintain the same metric moving forward, it is vital to realize that these figures are likely to change relatively quickly. As Dixon and colleagues (2015) correctly pointed out, the field of ABA is expanding at an exceedingly fast pace. In order to meet the very real need for our services in our communities, we are developing many new ABA programs and existing programs are often growing. As a result, universities are hiring many junior faculty members, most of whom are currently experiencing their first few years at the academy. Publication rates for beginning faculty can be expected to accelerate after they teach their first courses, acquire the skills to work effectively with their graduate assistants, and learn that the fastest way to get your research materials ordered is by showing the administrative assistant some respect. As these young scholars begin to publish more frequently in their third, fourth, and later years as professors, the figures identified by Dixon and colleagues are likely to change. Programs and faculty that now appear to be publishing at low rates may be publishing quite frequently by 2020.

We owe Dixon, Reed, Smith, and Belisle (2015) a debt of gratitude for starting an important conversation—a conversation that should be robust. My perspective on the scholarly work of ABA faculty is more favorable than the data presented by Dixon and colleagues (2015) for the aforementioned reasons. Only as we further explore this issue, will we be able to truly identify accurate metrics against which to assess our training programs. Perhaps only time and a broader vantage point will tell if the emperor is wearing pants.