Research Productivity Among Practitioners in Behavior Analysis: Recommendations from the Prolific

Applied behavior analysis (ABA), the application of the principles of behavior to problems of social significance, relies on peer-reviewed research to establish and confirm the effectiveness of practical procedures across a wide variety of applied areas (Baer et al. 1968). These areas include, but are not limited to, the assessment and treatment of behavior excesses among individuals with intellectual disabilities, skill acquisition among children with autism, organizational behavior management, brain injury rehabilitation, behavioral gerontology, and health and fitness (Fisher et al. 2011).

In recent years, an increase in the demand for behavior analytic services has resulted in a dramatic increase in the number of behavior analytic practitioners. For example, the number of individuals certified by the Behavior Analyst Certification Board (BACB®) has increased from just over 5000 in 2008 (Shook and Favell 2008) to over 18,000 in 2015. This demand for behavior analytic services has also generated new academic training programs in ABA. For example, the BACB now reports approved university course sequences at over 260 institutions worldwide. Despite this increase in training programs, however, the number of researchers in ABA may not be keeping pace. In fact, some reports suggest that the number of new researchers may be stable or even decreasing (Dymond et al. 2000). Dixon et al. (2015) reported that many faculty members in ABA programs have not published a single article in the field’s top journals. The relatively small number of ABA researchers in academic settings who are conducting research may be increasingly unable to produce the volume of information needed by practitioners to advance behavior analytic practice in many applied areas.

The field of clinical psychology faced similar hurdles in the early years of its development. One of earliest approaches to training clinical psychologists was the scientist-practitioner model, which was formally declared at the annual convention of the American Psychological Association in Boulder, CO, USA, in 1949. This model later became known as the “Boulder model” and emphasized training in research methods and the application of research findings to clinical problems. The logic behind the Boulder model was that the field needed research to identify effective assessment and treatment procedures and that students trained in this model would be well equipped to produce this research. However, as the number of clinical psychologists with a primary interest in practice increased, an alternative training model, the practitioner-scientist model, grew quickly (Peterson and Park 2005). This training model prioritizes the development of applied repertoires and the consumption of research over research production. Today, there are as many clinical psychologists trained in the practitioner-scientist model as there are graduates of scientist-practitioner programs (Norcross and Castle 2002). Most graduates of practitioner-scientist programs become clinicians, and many academic psychologists are still concerned that the field has too few researchers to empirically examine the many and varied challenges faced by practicing psychologists.

The Boulder model is considered by some to be a failure in clinical psychology because it has not produced an increase in practitioner-conducted research and, as many would argue, it has not resulted in practitioners who are immune to nonempirically supported interventions (Nathan 2000). This state of affairs is largely due to the differing contingencies in operation for scientists versus practitioners. Scientists (at least in academic settings) must publish to advance their careers and practitioners must see clients to earn a living. Seeing clients is often incompatible with conducting research, so most practitioners spend little time on research activities. Nevertheless, many argue that the central notion of the scientist-practitioner model should not be abandoned, as it represents the future of the discipline (Nathan 2000).

Similar developments appear to be playing out in ABA. The discipline has grown tremendously over the last decade, with practitioners accounting for the vast majority of this growth. Although the number of academic training programs has increased, most of these have been at the master’s degree level. The number of doctoral programs in ABA has not increased at the same rate (Critchfield 2011). Consequently, the number of doctoral-level researchers has remained relatively steady. As is the case in clinical psychology, most students in ABA are interested in practice careers. The contingencies favor this, as there are a large number of job opportunities for practicing behavior analysts and the income potential is quite good. Although this certainly has produced some benefits to society (e.g., increased access to services) and to the discipline (e.g., increased recognition by government and third-party payers), it has increased the number and variety of applied topics that need investigation by researchers and has created a gap between what practitioners are expected to undertake in practice and the available research literature on which to base their work.

One solution to this problem is for practitioners to conduct more applied research. After all, practitioners have one important advantage over academics when conducting applied research: They work in settings in which they have frequent contact with problems that may be of interest to applied behavior analysts. Of course, a number of barriers to conducting research exist for many practitioners. Individuals attempting to conduct research in applied settings may have few resources available to them, such as tools (e.g., data collection equipment), administrative support, and space. Notably, research must often be approved by a research review board, which can serve as a guide to the ethics of conducting research with clients. Unfortunately, review boards may only be accessible through a university or hospital. Furthermore, most practitioners are very busy and may have little time to devote to research. Finally, many employers may see little value in research and may not additionally compensate employees for time spent conducting research.

Another barrier to practitioner research productivity is that many journals require rigorous methodology and experimental designs for publication. These requirements may be difficult for practitioners to meet. Often, AB or “case study” designs are used in practice, and resources may not be available for assessment of interobserver agreement in many settings. Given these constraints, some practitioners and organizations may not attempt to conduct research at all.

Despite the obstacles described above, a number of ABA practitioners have consistently produced research. Some have even managed to remain productive over the course of entire careers. However, no studies have been conducted to identify these practitioners and, perhaps more importantly, to study their research-related behavior. A description of the activities in which productive practitioners engage may be helpful for other practitioners who aspire to increase their own research productivity. Thus, the purposes of the current study were to (a) identify the most prolific practitioner authors in the field of ABA and (b) generate a list of recommendations from these authors that may be helpful to individuals seeking to increase or maintain research productivity outside of academia.

Phase 1: Identification of Prolific Practitioner Researchers

Method

The purpose of phase 1 was to identify the most prolific practitioner researchers in ABA. All issues of the Journal of Applied Behavior Analysis (JABA), the Journal of Organizational Behavior Management (JOBM), and Behavior Analysis in Practice (BAP) published between January 2000 and December 2014 were examined. Authorship and affiliation information were procured from articles that met the following criteria: (a) The article included at least one research participant and (b) the article contained a method section, results section, and discussion section, although the results and discussion sections may have been combined. Discussion articles, book reviews, and technical articles without a method, results, and discussion section were excluded from the analysis as the focus of this study was on empirical articles.

Authors with nonacademic affiliations and institutions listed within the first six authors on publications in one of the three journals listed above between the years 2000 and 2014 were counted by providing each author with one publication credit per article (Shabani et al. 2004). Once counted, publication credits for all authors were summed. If more than one author from a particular institution was recorded, only one publication credit was given to that institution. Only the first six authors of each article were given credit and credit was given regardless of authorship order. An affiliation was considered nonacademic if it did not have the words “academy,” “college,” “institute,” or “university” in the name. However, some affiliations with the word “institute” are primarily nonacademic (e.g., May Institute); therefore, each affiliation with the word “institute” was examined individually to determine if the organization offered an educational degree (academic) or provided a clinical or organizational service (nonacademic). Organizations that are a part of a college or university (e.g., the Kennedy Krieger Institute, which is affiliated with Johns Hopkins University) were considered academic. If an author listed his or her affiliation as both a nonacademic organization (e.g., The New England Center for Children) and an academic organization (e.g., Northeastern University), an academic affiliation was recorded.

To analyze authors and their affiliations, we created a spreadsheet that was comprised of rows (one for each article) and columns (relevant data for each article). For each journal, beginning with the first issue published in 2000 and continuing through the last issue of the volume published in 2014, we recorded in separate columns the following for each article: journal; year; issue and page number; title, the first six authors/affiliations; whether the article was an empirical article with at least one participant; and whether it included method, results, and discussion sections. We considered the data in each column as a point for the purposes of intercoder agreement. After all articles had been coded, we eliminated articles that did not include at least one participant and a method, results, and discussion section. Next, we identified authors with nonacademic affiliations. We then summed the number of points for each author.

Intercoder Agreement

A second observer independently scored 25 % of the articles in each volume from the three journals. An agreement was defined as both observers recording the same volume number, title, authors, and institutions for each article in a journal for a specific year. Intercoder agreement was calculated by dividing the number of agreements by the number of agreements plus disagreements and converting the ratio to a percentage. Point-by-point agreement across the three journals was 98 % (range, 97 to 100 % across the three journals).

Results and Discussion

Overall, 1205 articles in three major ABA journals (JABA, JOBM, and BAP) were identified for the 15-year coding period. The majority of articles were published in JABA (83.6 %), while the remainder (12.5 and 3.9 %) were published in JOBM and BAP, respectively. These articles were written by 1851 authors from 565 institutions. A total of 328 (17.7 %) of the authors had nonacademic affiliations. A total of 485 of the 1205 (40.2 %) articles included at least one nonacademically affiliated author.

The ranking process resulted in the identification of the seven most prolific nonacademically affiliated authors (see Table 1). These authors are Dennis H. Reid, Marsha B. Parsons, James K. Luiselli, Jonathan R. Tarbox, Bridget A. Taylor, Perry L. Lattimore, and Louis J. E. Malenfant. A few findings regarding the information in Table 1 are worth highlighting. First, although the terminal practice degree in ABA is the master’s degree, the seven most productive practitioners have doctoral degrees; this suggests that training beyond the master’s degree may enhance research productivity for practitioners. The mean number of publications per year for each of these individual authors was .67 (range, .46 to 1.13) and the median was .53. The top seven most prolific practitioner authors accounted for 71 of the 485 (14.6 %) publications authored by practitioner authors.

Table 1.

The most prolific nonacademic authors in JABA, JOBM, and BAP from 2000 to 2014

Author name	Institutional affiliation	Total number of publications
Dennis H. Reid	Carolina Behavior Analysis and Support Center	17
Marsha B. Parsons	J. Iverson Riddle Center and Carolina Behavior Analysis and Support Center	16
James K. Luiselli	May Institute	9
Jonathan R. Tarbox	Center for Autism and Related Disorders, Inc.	8
Bridget A. Taylor	Alpine Learning Group	7
Perry L. Lattimore	J. Iverson Riddle Center and Western Carolina Center	7
Louis J. E. Malenfant	Center for Education and Research in Safety	7

In some ways, these data are encouraging. Over 40 % of articles in our database included at least one practitioner author, suggesting that ABA already benefits from a substantial proportion of practitioner-researchers. Alternatively, over 80 % of authors had academic affiliations, which might suggest a need for additional research involvement by practitioners. Future research should examine these data across varying time periods to determine if the percentage of practitioner authors is changing.

Of course, these data should be interpreted in context. Only empirical articles from three journals were analyzed; the discipline of ABA includes a number of other journals (e.g., Behavioral Interventions and The Analysis of Verbal Behavior), some of which might include a higher percentage of practitioner-authored studies. Future research could include additional journals when examining this topic. Also, nonempirical articles were excluded from the analysis. It is possible that some practitioners publish a number of nonempirical articles (e.g., practice guidelines) that might be included in future research. Further, we coded authors who listed an academic affiliation as academic even if their first listed affiliation was nonacademic in nature. We coded the data in this way because it was impossible to determine the amount of time and degree of affiliation that individual authors have with the academic organizations with which they are affiliated. It was also impossible to determine the contingencies arranged by these academic organizations. Although some authors may have a primarily nonacademic affiliation and simply maintain adjunct status with an academic institution, others may hold rank at their academic affiliation. Even if an author is not in rank, a secondary academic affiliation may influence research productivity by providing additional compensation or opportunities contingent upon publishing. Coding all authors with any listed academic affiliation as academic seemed the most prudent option, but future researchers may categorize authors differently. Future research might even examine an individual’s job description or responsibilities, as this information might be more revealing than a listed affiliation.

Phase 2: Recommendations for Aspiring Practitioner-Researchers

Method

The purposes of phase 2 were to determine the strategies and variables that may influence the research productivity of the authors identified in phase 1 and to generate a list of recommendations that might be of use to practitioners interested in conducting research. Six of the seven authors with nonacademic affiliations who received the most publication credits as determined in phase 1 participated in phase 2; one author was unable to participate. Each author was interviewed using a standard list of questions that pertained directly to their research-related behaviors. Research-related behaviors consisted of meeting with peers or supervisees about research, conducting literature reviews, collecting and analyzing data, and writing the results of research. Interviews were conducted individually via phone call and were audio-recorded. Twenty questions were used as a starting point for each interview, but these questions were subject to additions during the interview. The 20 questions asked of each interviewee are listed in Table 2.

Table 2.

Summary of interview responses from prolific practitioner authors

Question	Response	Number of authors who responded
What is your motivation to conduct research?	Improve service delivery	3
How do you develop research ideas?	Issues that arise during practice	3
How frequently do you meet with other researchers?	Monthly	2
How do you meet with research peers/supervisees?	In person	6
How do you collect data?	Paper and pencil	5
Who are your data collectors?	Front-line staff	3
Are data collectors compensated in any way?	No	5
How often do you meet with data collectors?	Weekly	3
What program do you use to graph your data?	Microsoft Excel™	3
Do you graph the data or do you have assistance?	Have assistance	3
When do you write?	Whenever there is time	3
How often do you write?	Weekly	3
What is the greatest barrier to your writing?	Time	4
What is the greatest facilitator of your writing?	Accountability to others	2
Do you receive extra compensation for the research that you generate?	No	4
Who submits the manuscript for publication?	Interviewee	5
Who is responsible for reviewer changes that are requested?	Interviewee	2
Do you set goals for yourself? If so, how often?	Yes, number of projects per year	3
Does the organization that you work for encourage you to conduct research?	No, there is not much support/encouragement	3

Responses to the questions were compiled for each of the six authors and used to create a list of recommendations. In order for a response to qualify as a recommendation, the response must have been nominated by four or more of the six authors. For example, if four authors recommended setting daily writing goals, then setting a daily writing goal was included as a recommendation. In addition, the response had to be relevant for a recommendation. For example, although four of the six authors reported not being compensated for conducting research, we did not recommend that practitioners refuse compensation for research.

Interobserver Agreement

A second observer independently listened to a random selection of 25 % of the questions asked across the six interviews. A pre-developed interobserver agreement (IOA) form listing each question and four possible answers was used to determine agreement. Only one of the four answers for each question matched the interviewee’s answer. This form was used to avoid requiring a second observer to transcribe the audiotape word-for-word. IOA was calculated by dividing the number of agreements by the number of agreements plus disagreements and converting the ratio to a percentage. Point-by-point agreement was 100 %.

Results and Discussion

Results from interviews with six of the seven most prolific practitioner authors, as identified in phase 1, are summarized in Table 2. Each question, followed by the number of interviewees who responded in the same manner (the specific response is listed in the middle column), is provided. A number of the findings are noteworthy. For example, half of the authors reported that their ideas for research are derived from issues that arise in practice. This is encouraging, as practitioners may be in a unique position to identify important research topics that academic researchers may not contact on a daily basis. Half of the authors also reported using front-line staff as data collectors; other practitioners may also be able to encourage front-line or direct-care staff members to assist them with data collection. Interestingly, the majority of authors reported that their staff members are not paid to collect data. Many of the authors reported working on research weekly, and three reported that they write on a weekly basis. Many authors reported that they use Microsoft Excel™ to display and graph their data, and three authors reported setting formal goals to accomplish research-related tasks.

A number of authors appear to be motivated to conduct research primarily by improving their daily practice. In addition, four of the six authors interviewed do not receive additional compensation for producing research and three of the six reported that they do not even receive formal encouragement to conduct research from the organizations for which they work in. This is discouraging, as it suggests that some employers find relatively little value in conducting applied research.

Of the 20 questions posed to each interviewee, 5 of these met the requirements to qualify as a recommendation (see Table 3). That is, four or more of the six authors provided the same response for 5 of these 20 questions, and that response was relevant for a recommendation. The first recommendation was also the answer to the only question in which all six authors responded identically. All authors reported meeting with peers and supervisees to plan or conduct research “in person.” Although this does not necessarily mean that phone or video conferencing is a poor way to plan or conduct research, shoulder-to-shoulder planning, data collection, and data interpretation may be a particularly effective way for practitioners to accomplish research-related tasks. If practitioners work in the same setting in which direct-care staff collect data, in-person meetings should be relatively easy to accommodate. The second recommendation was in response to the question “How do you collect data”? Five of the six authors reported that they use paper-and-pencil methods of data collection. Of course, this does not mean that digital methods of data collection are inferior; it is possible that many applied settings do not use tablets, phones, or laptops for data collection due to their expense and the possibility that they might be misplaced or damaged. Practitioners should use the most economical and feasible method of data collection for their setting. The third recommendation related to writing or reporting research findings. Four of the six authors noted that they write collaboratively. This is good practice, as it provides multiple perspectives on the topic and often enhances the readability of manuscripts. The fourth recommendation related to the largest barrier to conducting and writing the results of research, which was reported by the authors to be a lack of time. Practitioners interested in conducting research should attempt to protect some time on a regular basis (weekly or monthly) to devote solely to research. The final recommendation relates to the person submitting the manuscript for publication. Five of the six authors interviewed reported that they submit their own manuscripts for publication. Authors who are most successful at producing research may be more likely to closely monitor the project from the beginning stages (e.g., literature review) through manuscript submission to a professional journal.

Table 3.

Recommendations for practitioners aspiring to increase their research productivity

1. Arrange as much face to-face contact with your peer researchers, collaborators, and research supervisees as possible.

2. Use an economical, reliable, and safe method of data collection; if possible, have staff members assist in data collection.

3. Write the results of your research collaboratively.

4. Dedicate time to plan, conduct, or write the results of research on a regular basis (ideally each week).

5. Stay close to your project from the initial planning stage through journal submission.

It should be noted that the recommendations generated from this study may be helpful to practitioners aspiring to increase their research productivity if the practitioner already has a reasonable research-related repertoire which was acquired in graduate school. Practitioners without this repertoire may be unable to conduct research despite the recommendations provided in this study and might consider obtaining additional training in research methods.

General Discussion

Many behavior analytic practitioners do not conduct research. This is not surprising as the contingencies in practice environments do not naturally support such activity. However, due to the rapid growth of the field and the many unanswered applied research questions, it is important for the discipline to develop additional research capacity. Fortunately, the authors identified in our analysis are positive examples of ongoing research activity in practice environments.

In an effort to minimize the methodological constraints that may prevent practitioners from conducting and publishing research, some journals in ABA, such as JOBM and BAP, have created a new submission category entitled “reports from the field” and “brief practices,” respectively. The publication criteria for manuscripts submitted under these categories are generally more flexible, increasing the likelihood of their acceptance. More data are needed to examine the effects that these practitioner-friendly practices have had on nonacademic research productivity, but the adoption of this practice may very well have the intended effect of increasing submissions from practitioners. Practice-oriented journals, such as Behavior Analysis in Practice, have also appeared in recent years, perhaps making publication more attractive to practitioners.

The contingencies that support research productivity in academic settings are quite different than those which support research productivity in practice settings. In academic settings, individuals generally receive promotions, tenure, and in many cases more opportunities to travel and present their data contingent on publishing research. Practitioners generally do not experience these same contingencies. In fact, research in practice settings may result in additional work and little acknowledgement. Until the contingencies operating on practitioners change, practitioner research productivity may not dramatically increase.

Graduate training programs might help to increase research productivity in practice settings by offering a course, or at least a portion of a course, focusing on educating and training students on best practices for conducting research in applied settings. In addition, research-like behavior (e.g., careful measurement, IOA assessment, and experimental designs) can be modeled and reinforced in practice. Continuing education seminars or workshops on this topic might also be beneficial. Of course, the greatest increase in research productivity would likely be realized if nonacademic organizations began providing more acknowledgement, time, and financial incentives for publishing research. Although some large service providers in ABA (e.g., the Center for Autism and Related Disabilities, Inc., in Los Angeles) have begun doing this, large-scale adoption of this practice has not occurred, and its sustainability has not yet been demonstrated. Service providers who employ practitioners who conduct research may benefit from this arrangement via enhanced clinical reputation, which may result in additional referrals and business.

There are several topics related to practitioner research productivity for future researchers to explore. First, future researchers could conduct a similar study but examine academic authors. It may be interesting to examine the responses that academic researchers provide to the questions in phase 2 and compare these responses to practitioner researchers. Future research could also survey practitioners in a variety of applied organizations to inquire about the variables preventing research among employees and what could be done to increase research productivity among practitioners.

In conclusion, research productivity among practicing behavior analysts is an important topic in need of additional study. A small number of practitioners have managed to consistently publish high-quality peer-reviewed research for many years. This and future studies examining the behaviors in which these practitioner-researchers engage to maintain this productivity are warranted to more fully understand, and hopefully promote, much-needed additional research by practitioners.