Teaching Graduate Students to Translate Nonbehavioral Treatments Into Behavioral Principles

Abstract

Behavior analytic translations of nonbehavioral treatments were recommended by Brodhead in Behavior Analysis in Practice, 8(1), 70–78 (2015) as part of a decision-making framework for practitioners working on interprofessional treatment teams. Professionals from different disciplines often have overlapping scopes of practice and competence, yet each recommends interventions according to their discipline-specific worldview and training. Nonbehavioral treatment recommendations may be especially challenging for behavior analytic practitioners who are committed to the science of human behavior and who are also ethically obligated to work cooperatively and in the best interest of the client. Learning to translate nonbehavioral treatments into behavior analytic principles and procedures may function as a valuable means of improving professional judgment, thereby promoting evidence-based practice and effective collaboration. Behavioral translations may expose procedures that are, in fact, conceptually systematic, creating more opportunities for behavior analysts to partner in interprofessional care. Using a behavioral skills training package, graduate students of applied behavior analysis were taught to translate nonbehavioral treatments into behavior analytic principles and procedures. All students produced more comprehensive translations following training.

Evidence-based practice is emphasized across fields of medicine, education, rehabilitation, and allied health as a means of bringing scientific research into clinical practice. It has become an industry “buzzword” representing the most efficacious approach to client care. Applied behavior analysis has described evidence-based practice as the integration of its three defining principles: empirically supported treatments, client values, and professional judgment (Slocum et al., 2014; Smith, 2013). These three tenets closely align with the regulations outlined in the Ethics Code for Behavior Analysts under which we are obligated to use scientifically validated treatments, value our clients’ rights and choices, understand the limits of our own expertise, and continue to enhance our professional abilities (Behavior Analyst Certification Board [BACB], 2020). As such, synthesizing these three defining features promotes ethical conduct in applied practice.

Ideally, a model of evidence-based practice functions as a decision-making framework for practitioners to select and implement the most effective treatments and successfully address socially important issues (Slocum et al., 2014). However, professional judgment is central to the efficacy of this model and any weaknesses in professional judgment would risk compromising the entire framework. Behavior analysts exercising poor judgment may engage in unethical practices and fail to properly discern ineffective treatments. It becomes critically important for behavior analytic practitioners to develop the clinical wisdom necessary to advocate for scientifically supported treatments, especially in interprofessional settings where nonbehavioral recommendations are commonly accepted.

Behavior analysts frequently work in contexts that necessitate interprofessional collaboration, particularly in intervention with individuals on the autism spectrum (Botha et al., 2021), and the evidence-based practice model may enhance the cooperative efforts of the individual practitioners (Slocum et al., 2014). Professionals who provide services to individuals on the autism spectrum such as speech-language pathologists, psychologists, and occupational therapists, have adopted similar definitions of evidence-based practice (American Speech-Language-Hearing Association, 2005; Bennett & Bennett, 2000; Spring, 2007). This conjoint perspective creates a universal language and fosters a team culture that seeks the best available research in response to client needs (Slocum et al., 2014). Thus, a model of evidence-based practice promotes a unified, systematic approach to interprofessional care.

Despite this shared vision and common framework, behavior analysts working in collaborative contexts may diverge from the principles of evidence-based practice. Schreck et al. (2016) surveyed 876 behavior analysts regarding their use of common autism treatments with varying levels of scientific support. While most respondents reported using empirically validated treatments, others claimed to use unestablished interventions such as sensory integration therapy and auditory integration training.  More alarmingly, a small percentage of respondents reported using treatments found to be ineffective and even harmful such as facilitated communication. An examination of the factors influencing the selection and continued use of these treatments found that “persuasion to conform” (e.g., recommendations from supervisors, colleagues, instructors, employers) significantly influenced behavior analysts’ decisions across all levels of scientific evidence—established, emerging, unestablished, and ineffective/harmful (p. 372).

Collaboration with others, particularly those from other disciplines, presents tremendous professional development opportunities for behavior analysts and can lead to enhanced outcomes for clients. Collaboration can also create confusion for the behavior analyst, as other members of the team may suggest interventions that are not empirically supported. The influence of opposing worldviews and different standards of evidence may encourage a divergence from the ethical code and the scientific principles of human behavior. Nonbehavioral treatment recommendations impose a responsibility for behavior analysts to balance the multifaceted aspects of evidence-based practice and ethical care to effectively analyze the proposed intervention. Hence, interprofessional collaboration requires behavior analysts to not only be grounded in the scientific evidence for a given procedure, but to display an astute level of professional judgment.

The principle of professional judgment may be conceived as a point of convergence where empirically supported treatments intersect with client values and context. Any attempt to highlight the intricacies of professional judgment would be trivial at best, as the specific responses which may be encompassed under this tenet are undoubtedly extensive and complex. Perhaps the essence of professional judgment is best captured in the analytic nature of our discipline and our reliance on conceptual systems (Baer et al., 1968, 1987). These dimensions of applied behavior analysis were exemplified in Leaf et al.’s (2016) description of highly qualified practitioners. As explained, truly effective, skilled practitioners develop and implement procedures that are conceptually consistent with the scientific principles of human behavior. They continuously analyze the impact of their interventions and other critical factors influencing behavior change. These expert behavior analysts appreciate the “why” of their interventions. More precisely, they understand the theoretical basis for their procedures and can relate the success and failures of their treatments to the basic principles and scientific methods. On the contrary, an insufficient understanding of the experimental and philosophical doctrine results in what Michael (1980) called “superficial behavior analysis” (p. 14). The dangerous result is an inability to interpret the complexities of human behavior, and such “analytic superficiality” (p. 14) consequently impedes the behavior analytic worldview.

We are obliged to use interventions that are rooted in science and align with the concepts and principles of behavior analysis (BACB, 2020, Code 2.01; Baer et al., 1968, 1987; Weiss, 2018) all the while working cooperatively with professionals from other disciplines and in the best interest of the client (BACB, 2020, Code 2.10). In the case of nonbehavioral treatments, these ethical responsibilities may seem incompatible and may complicate interprofessional conversations and treatment decisions. Questioning or rejecting the intervention in the name of behavioral science risks eroding the professional relationship and terminating the collaborative approach (Brodhead, 2015). On the other hand, accepting the treatment to be amiable and respectful may result in the use of an ineffective or even harmful intervention. Each of these circumstances threatens effective, interprofessional collaboration and the evidence-based model of care.

As other professionals propose nonbehavioral treatments we must evaluate the potential outcomes and consider the impact on the current interventions and overall program objectives while staying true to the science of behavior analysis and ethical practice (BACB, 2020, Codes 2.01, 2.14, 2.18; Brodhead, 2015; Newhouse-Oisten et al., 2017). Such judgment requires a thorough analysis of the controlling variables and a conceptual translation of the nonbehavioral treatment. According to Brodhead (2015), by translating the nonbehavioral treatment into behavioral principles, the behavior analyst can adequately assess its potential effect. Professionals from other disciplines may use different terminology and accept alternate theories or explanations for human behavior, but a thorough analysis of the nonbehavioral treatment, according to a behavior analytic worldview, may reveal conceptual similarities. That is, a behavioral interpretation of a nonbehavioral treatment may impart its conceptual sense and success in practice. Otherwise, the behavioral perspective exposes its inconsistencies with the science of behavior and informs the practitioner’s response to the proposal.

As a simple example, consider an interprofessional team including a Board Certified Behavior Analyst (BCBA), speech-language pathologist, occupational therapist, physical therapist, psychologist, and special education teacher. A member of the team recommends sensory integration therapy to address motor stereotypy, which is interfering with the client’s participation in therapeutic and educational activities while seated at the table (see Levinson & Reid, 1993; Oriel et al., 2011). The colleague suggests the client engage in proprioceptive activities such as running or jumping just prior to completing work at the table. From the colleague’s perspective, the stereotypy is the result of an underlying sensory integration disorder, and the running will provide the necessary input to regulate sensory processing. Initial considerations of this intervention would raise concerns about the opposing theories of behavior and lack of scientific support for sensory integration treatments (see Zimmer & Desch, 2012). However, a behavioral translation of the treatment could interpret the running and jumping as antecedent exercise that may have an abolishing effect on the motor stereotypy (see Neely et al., 2014). Here, the BCBA and the nonbehavioral colleague differ in their understanding of the mechanisms responsible for the effects of the intervention but may be able to work collaboratively and share the responsibilities of treatment.

Brodhead’s (2015) recommendation to translate nonbehavioral treatments into behavioral principles was presented as part of a decision-making framework to help practitioners systematically evaluate nonbehavioral treatments presented in collaborative contexts but has not yet been empirically tested. Under this model, behavior analysts presumably learn more about the treatment, come to understand their colleague’s perspective, and determine when it may be necessary to express concerns. Brodhead’s model helps to expand the consideration of interventions and emphasizes an analysis of the expected outcomes. Discussions about the acceptance of a proposed treatment pose some potential interference within the collaborative partnership. Such action is only advised when the nonbehavioral treatment compromises the client’s safety or interferes with the current treatments and/or objectives. According to the model, once a nonbehavioral treatment has been identified, the behavior analyst first considers risk of physical and/or psychological harm. Any threats to the client’s safety would, of course, require the behavior analyst to immediately address the treatment with the nonbehavioral colleague. Barring any initial safety concern, the behavior analyst seeks a thorough understanding of the treatment by reviewing the available research and consulting professionals from the prescribing discipline. With this new information, client safety is reassessed and the nonbehavioral treatment is translated into behavioral principles. If the translated intervention seems likely to have a positive impact, then as Brodhead explained, implementation can proceed. Then again, if success is unlikely, the behavior analyst must evaluate the degree to which the nonbehavioral treatment will interfere with the current goals and consider if this impact is sufficient to justify addressing the treatment and potentially jeopardizing the professional relationship. Translating nonbehavioral treatments into behavioral principles as recommended by Brodhead (2015) functions as an effective aspect of professional judgment for evidence-based practitioners. This valuable skill embodies the conceptual analysis vital to the behavior analytic worldview and promotes an ethical approach to interprofessional collaboration. Therefore, the purpose of this study was to examine the effects of a behavioral skills training package on graduate students’ ability to generate a behavioral translation of nonbehavioral treatments. Very little exists about teaching complex skills involving interprofessional collaboration to students of behavior analysis and it represents a contribution to the literature in this area as well. This study sought to concretely teach the translational skill set to students of behavior analysis facing dilemmas regarding the integration of nonbehavioral interventions.

Method

Participants

The study included four graduate students attending an online master’s program in applied behavior analysis. All students had completed at least three semesters in the graduate program, carried an overall graduate-level grade point average of at least 3.5, and earned a minimum grade of B + in a prior course covering the basic concepts and principles of applied behavior analysis. They participated in the training activities described here in partial fulfillment of the requirements for a graduate class, Effective Collaboration Across the Disciplines. The participants each consented to the use of their data for the purposes of the current study. Course grades were not contingent on such consent.

Demographic information can be found in Table 1. As part of their current clinical experience, all participants reported that they provided behavior analytic services and shared clients with professionals from other disciplines. Courtney chose not to report any additional demographic information or experience with cross-disciplinary collaboration. Although Nick, Jackie, and Cody all worked in interprofessional settings, their collaborative practices varied greatly. The specific disciplines with whom they collaborated were quite diverse and included professionals from allied health, medicine, education, and administration. The frequency of collaborative interactions ranged from biweekly to monthly and the extent of cooperation varied from more autonomous approaches where professionals worked in parallel (Cody), to independent assessment but shared goal development and treatment (Nick), to joint responsibility in all aspects of care (Jackie).

Table 1.

Participant Demographics

Student	Age (years)	Gender	Race/Ethnicity/Origin	Clinical experience (years)
Courtney	NR	Female	NR	NR
Nick	30	Male	Hispanic/Latinx/Spanish	3
Jackie	31	Female	Hispanic/Latinx/Spanish	9
Cody	26	Male	White	3

Note. NR = Not reported

Settings and Materials

All materials including video models, case scenarios, and worksheets were made available via a learning management system used for online instruction. Students submitted responses using the file upload or quiz feature within the system and were able to view written instructor feedback in the assignment comments or annotations. A web conferencing tool was used to host individual meetings with students as needed to further discuss instructor feedback. Such meetings occurred on three occasions with Courtney and once with Jackie.

Case Scenarios

Students were presented written passages depicting fictitious clinical vignettes with recommendations for nonbehavioral treatments (see Appendix A). These case scenarios portrayed the student as the behavior analyst working on an interprofessional treatment team and described the client, relevant background information, applied setting, behavior(s) of concern, the nonbehavioral colleague’s recommendation for treatment, and the expected outcome. All treatments were described from the nonbehavioral colleague’s perspective.

Client information typically included name, age, gender, diagnosis, treatment context, behavioral goals, other therapies, and current concerns. Some scenarios also included preferences, aversions, assessment results, recent progress, and current level of functioning. In all case scenarios the fictitious clients carried a diagnosis of autism spectrum disorder. The nonbehavioral treatments were recommended to address either the core symptoms of autism (e.g., functional communication, social interactions, motor stereotypy), common comorbidities (e.g., food selectivity, sleep disorders, maladaptive behaviors), or other developmental conditions (e.g., dysgraphia, stuttering, articulation impairments). A total of 20 case scenarios were created featuring treatments from the fields of speech-language pathology, occupational therapy, physical therapy, education, art therapy, music therapy, psychology, and medicine (see Table 2). Descriptions of the nonbehavioral treatment procedures were taken from scholarly articles and discussion papers published in discipline-specific journals or written by authors within the field and published in more general sources. Although many of the treatments included in the case scenarios span the practice purviews of multiple professions, the specific discipline was assigned based on the authors, procedures, and source. For example, handwriting skills may be addressed by both teachers and occupational therapists. The handwriting treatment here was presented as a recommendation from a teacher because the treatment procedures were primarily taken from an article published in the Journal of Educational Psychology and intended to improve handwriting instruction in the classroom. A list of the case scenario, nonbehavioral treatment, associated discipline, and primary source are provided in Table 2. Students in the training group were assigned all 20 case scenarios to translate the nonbehavioral treatments and additionally had access to the three model scenarios.

Table 2.

Case Scenarios With Corresponding Treatments and Sources

Case scenario	Treatment	Associated discipline	Source
1	Work system	Education	Hume & Odom (2007)
2	Therapy ball chairs	Occupational therapy	Bagatell et al. (2010)
3	Build-A-Face	Art therapy	Richard et al. (2015)
4	Social communication	Speech-language pathology	Timler et al. (2005)
5	Qigong massage	Occupational therapy	L. M. T Silva et al. (2008, 2011)
6	Lidcombe program	Speech-language pathology	Onslow et al. (2020) Wagaman et al. (1993)
7	Instructional approaches for handwriting	Education	Berninger et al. (1997)
8	Individually composed songs	Music therapy	Kern et al. (2007)
9	Nonspeech oral motor exercises	Speech-language pathology	Guisti Braislin, & Cascella (2005) Forrest & Iuzzini (2008)
10	Sensory adapted dental environment	Occupational therapy	Cermak et al. (2015)
11	Aerobic exercise	Physical therapy	Levinson & Reid (1993) Oriel et al. (2011)
12	Yoga	Physical therapy	Kaur & Bhat (2019)
13	Animal assisted therapy	Psychology	Martin & Farnum (2002) K. Silva et al. (2011)
14	Visual imagery	Occupational therapy	Precin (2010)
15	Chronotherapy	Medicine	Czeisler et al. (1981)
16	Food chaining	Speech-language pathology	Fishbein et al. (2006) Castro (2018)
17	Visual support	Speech-language pathology	Johnston et al. (2003)
18	Power cards	Education	Davis et al. (2010)
19	Simple music	Music therapy	Kalas (2012)
20	Art activity	Art therapy	Kuo & Plavnick (2015)
Model 1	Sensory activities	Occupational therapy	Bailey & Burch (2016, p. 247)
Model 2	Counting-on strategy	Education	Cihak & Grim (2008)
Model 3	iPad play story	Speech-language pathology	Murdock et al. (2013)

Note. The model scenarios were used by the instructor to model the analysis and translation. They were not assigned to students.

Worksheet

The worksheet was a training tool designed to help students analyze the nonbehavioral treatment and translate it into behavioral principles and procedures (see Appendix C). The worksheet contained 40 questions divided into seven blocks each analyzing different components of the treatment: target behavior, antecedent variables, consequence variables, verbal behavior, and behavior change procedures, which made up the final three blocks. Each block contained four to eight questions. Questions were primarily based on the concepts, principles, and procedures in the fifth edition BCBA/BCaBA task list (BACB, 2017) and the descriptions provided by Cooper et al. (2020). These questions were intended to evoke private and/or overt intraverbal behaviors that would function as mediating responses to more complex verbal translations of the nonbehavioral treatment. The worksheet also included a checklist of principles and procedures for students to evaluate their written translation and confirm that any relevant components were included.

Video Models

A total of three video models were recorded and uploaded to the online learning management system for student access. Each video modeled a behavior analytic translation of a nonbehavioral treatment. The video was recorded using a screenshare feature which allowed for simultaneous presentation of the course instructor and the materials (e.g., case scenario, model worksheet, narrative translation) during the recording. The instructor presented a novel case scenario (different from that assigned to the students), analyzed the nonbehavioral treatment using the worksheet, and composed a behavioral translation.

Scoring Rubric

A scoring rubric was developed to assess student translations and ensure they were correct, thorough, and included essential components. The rubric was not distributed to students but used by the researchers to measure performance. It was created according to the concepts, principles, and procedures described by Cooper et al. (2020) and contained in the fifth edition BCBA/BCaBA task list (BACB, 2017). It included a list of 39 behavioral principles and procedures that corresponded to the questions on the worksheet (see Table 3). Using the rubric, each case scenario was separately analyzed by the lead author and two additional BCBAs. All expert raters were doctoral students of behavior analysis with an average of 8 years of clinical experience and interprofessional collaboration. The raters were instructed to review the case scenario and indicate the components that should be included in the translation to analyze the treatment according to a behavior analytic perspective. In other words, the expert raters selected the items on the rubric that they deemed essential to a behavioral translation of the nonbehavioral treatment depicted in the case scenario. They were permitted to list additional principles and/or procedures that they believed necessary but were not included on the list, however this never occurred.

Table 3.

Essential Components for Each Case Scenario

Components	Case Scenarios
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20
Overall goal of the intervention	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
Responses targeted for change	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
Covert responses				x								x		x
Overt responses		x		x	x	x	x	x	x	x	x	x	x	x		x	x	x		x
Operational definition of target behavior(s)	x			x	x	x		x	x	x	x	x		x		x	x	x	x	x
Function of the behavior	x	x						x		x	x		x			x	x			x
Classification of verbal operants				x		x						x					x	x
Antecedent stimuli	x	x			x	x	x	x		x	x		x	x		x	x	x	x	x
Preference towards antecedent stimuli		x								x	x		x			x			x
Aversion towards antecedent stimuli	x				x								x			x
Discriminative stimuli for reinforcement	x		x			x					x			x	x		x	x
Discriminative stimuli for punishment
Value altering effect of motivating operation		x								x	x		xa		x	x				x
Behavior altering effect of motivating operation		x									x		xa		x	x	x			x
Stimulus and/or response prompts	x	x	x	x		x	x	x	x		x	x		x			x	x	x	x
Consequence stimuli	x	x	x	x		x	x	x	x		x						x		x
Preference toward consequence stimuli	x		x				x				x						x
Aversion toward consequence stimuli
Positive reinforcement	x		x	x		x	x	x			x						x		x
Negative reinforcement
Positive punishment				x
Negative punishment
Extinction					x			x
Description of three term contingency	x			x		x		x			x								x
Stimulus–stimulus pairing										x								x
Premack principle	x
Discrimination training			x			x
Noncontingent reinforcement													xa							x
Functional communication training
Differential reinforcement						x	x
Errorless learning														x
Behavioral skills training				x
Modeling and imitation				x		x	x		x			x					x
Observational learning				x								x
Shaping				x		x									x	x
Stimulus fading	x			x			x							x	x	x
Task analysis							x	x						x				x
Chaining				x			x	x						x
Self-management	x			x		x								x				x
Total	15	10	8	18	7	16	13	13	7	9	15	9	8/9	13	7	12	14	11	9	10

^aThe expert raters interpreted these antecedent changes as establishing operations or noncontingent reinforcement. All expert raters for this case scenario agreed that either response was acceptable.

Historically, expert panels have been employed to evaluate validity and reliability of criterion-referenced assessments (Padilla & Akers, 2021; Usry et al., 2018), visual inspection and analysis (Hagopian et al., 1997; Roane et al., 2013), and to identify elements of intervention models (Callahan et al., 2010; Frost et al., 2020). Expert raters were used in the current study to establish content validity for the scoring rubric for each case scenario. While reliability does not equal validity, according to Lawshe (1975), a strong consensus among expert raters is a clear indication that a particular component is “essential” or “not essential” (p. 567). When agreement is low, however, validity may be more difficult to determine. Lawshe noted that any items identified by most expert raters (more than 50%), carries some level of content validity. Thus, all items indicated by at least two of the three expert raters in the current study were deemed “essential” and included in the scoring rubric for the corresponding case scenario. Components selected by one or fewer expert raters were marked as “not essential” to a translation of the treatment depicted in the case scenario. For example, a scenario that recommended an antecedent diet change to decrease socially mediated motor stereotypy would not require a discussion of punishing consequences. However, a recommendation to regulate proprioceptive sensory input by having the client engage in heavy work tasks following tantrum behavior would need to consider punishing effects. As such, the type and total number of components essential to a behavior analytic translation varied across the case scenarios. The number of essential components ranged from 7 to 18 with a mean of 11.2. Table 3 displays the essential components for each case scenario. Some principles and procedures listed on the rubric were never deemed essential to any of the nonbehavioral treatments depicted in the case scenarios, namely, discriminative stimuli for punishment, aversion toward consequence stimuli, negative reinforcement, negative punishment, and functional communication training.

Dependent Measure

The dependent measure was the percentage of essential components correctly described in each student’s narrative translation of the nonbehavioral treatment depicted in the case scenario. The essential components, as determined by the expert raters, was listed on the scoring rubric for each scenario. The experimenter reviewed the student’s narrative translation and scored “yes” or “no” for each essential component listed on the rubric. A score of “yes” indicated the component was correctly explained in the translation whereas “no” indicated the component was omitted or incorrectly described. The total number of “yes” scores was divided by the total number of essential components and multiplied by 100 to calculate the percentage correct. Percentage correct was the most appropriate measurement because all components were not applicable to all scenarios.

Translations

The narrative translation was a behavioral interpretation of the procedures described in the nonbehavioral treatment. By analyzing the controlling variables and identifying elements that were conceptually similar or inconsistent with the principles and procedures of behavior analysis, students could evaluate the potential effects of the recommended treatment (see Brodhead, 2015). While the outcomes of the intervention could not be precisely determined, reasonable speculation was encouraged. That is, the perceived function of environmental variables and the predicted effects of the procedures were to be included in the translation. For example, it is not possible to know that a certain consequence will have reinforcing effects, until that consequence is applied. However, it would be reasonable to predict that a favorite candy delivered contingent on writing the correct answer to a multiplication problem would increase such a response. Thus, the translation included the expected effects based on the information provided about the client and the interaction of the variables reported in the proposed intervention. It was possible that the behavioral translation exposed potentially unintended effects. For example, in case scenario 8 a music therapist recommended an individually composed song and picture icon to improve a greeting routine and decrease tantrum behavior. A behavioral translation of this treatment identified an extinction procedure which may temporarily increase the duration and/or magnitude of the tantrum behaviors. Such an effect may be undesired given the context yet would be important to a behavioral translation and evaluation of the proposed treatment.

Interobserver Agreement and Procedural Integrity

The experimenter scored all student translations across all conditions. A second rater reviewed 33.33% of Courtney’s translations, 33.33% of Nick’s translations, 33.33% of Jackie’s translations, and 36.00% of Cody’s translations across baseline, training, and posttraining. Interobserver agreement was calculated by dividing the number of rubric items in agreement by the total number of rubric items and multiplying by 100. Specifically, agreement was defined as both observers scoring “yes” to indicate a single, essential component was correctly described in the translation or “no” to indicate that it was not. A correct description included the specific aspect of the nonbehavioral treatment and a behavioral explanation. For example, in case scenario 16, a speech-language pathologist recommended the client’s favorite food, French fries, be systematically modified to increase consumption of a nonpreferred food. The narrative translation would include a brief reference to this aspect of the treatment and the behavioral interpretation (e.g., “The treatment includes a stimulus fading procedure whereby the texture and presentation of the French fries are gradually changed over time. The French fries are faded out while the mashed potatoes are faded in.”). Incorrect descriptions summarized the nonbehavioral treatment procedures without noting the behavioral principle or procedure (e.g., “The speech-language pathologist will change the appearance and texture of the French fries, so the client learns to eat new foods.”), or simply listed the behavioral principle or procedure without identifying the specific aspect of the treatment that it referred to (e.g., “The treatment will use stimulus fading.”), or completely omitted any reference to the procedures or essential component.

Agreement for Courtney, Nick, Jackie, and Cody was 86%, 82%, 84%, and 88% respectively. Cohen’s kappa was calculated to assess the significance of agreement between observers, beyond what might occur by chance (Watkins & Pacheco, 2000). The kappa coefficient was 0.672 indicating substantial agreement. The values reflect the scoring of 71–158 essential components per participant, each embedded in written, verbal narratives, with an average of 10–12 different components per scenario. Although this level of agreement is generally accepted (Kazdin, 2011), interpretation should also consider the complexity of the verbal behavior being measured.

Procedural integrity was assessed for all video models and the feedback provided on 34% of the student translations completed during the training condition. A second rater reviewed the recorded video models and collected data on the experimenter’s behavior using a checklist of the procedural steps. Scoring rubrics were compared to the written instructor feedback provided in the comments and annotations within the learning management system to ensure feedback was provided for all errors in the translation. Procedural integrity was 100% for video models and 97% for feedback.

Experimental Design

A multiple baseline design across participants was used to evaluate the effect of behavioral skills training on graduate students’ ability to translate nonbehavioral treatments into behavioral principles and procedures.

Experimental Conditions

Baseline and Posttraining

Case scenarios in the baseline and posttraining conditions were presented using the quiz feature in the online learning management system. Students read the case scenario and provided a written translation of the nonbehavioral treatment. Students were instructed to use behavior analytic terminology and translate the recommended treatment into behavioral principles and procedures to evaluate its potential effects. Feedback was not provided. Students were required to work independently and were instructed not to use any outside materials such as textbooks, journal articles, internet sources, etc. In addition, students completing scenarios in the posttraining condition were not permitted to use their worksheet. Posttraining sessions occurred when the student earned 80% correct or higher on two consecutive case scenarios during the training condition. One booster training session was provided when student performance fell below 80% for two scenarios in the posttraining condition. During the booster session students were able to use the worksheet to analyze the case scenario and received feedback on their translation with the opportunity to make revisions until they correctly described at least 80% of the essential components.

Translations submitted in baseline were graded based on completion rather than correctness. It was anticipated that student translations in baseline would be incomprehensive, and grades based on correctness would impose undue penalties. Therefore, all students received full points for the translations completed in baseline. Grades for translations in the posttraining condition were, however, based on correctness. This contingency was intended to evoke thorough and correct translations following training.

Training

Following a behavioral skills training model, training began with an introductory video providing a vocal description of the target skill and a rationale for its importance. Professional judgment as a tenet of evidence-based practice was discussed, including our ethical commitments to make clinical decisions based on the available science and the principles of behavior analysis (BACB, 2020, Code 2.01), to appraise the effects of treatments that may affect behavior change goals and programs (Code 2.18), and to advocate for our client’s right to effective treatment (Code 3.12). In addition, the detrimental effects of noncollaborative practices as described by Brodhead (2015) were discussed and his decision-making model for evaluating nonbehavioral treatments was presented and reviewed with particular emphasis on translations to identify conceptual similarities with the science and technologies of behavior analysis.

Next, students were provided a case scenario, worksheet, and access to the first video model. The video model began with the instructor reading a novel case scenario (different from the student’s case scenario), introducing the worksheet, and analyzing the nonbehavioral treatment by responding to the questions on the worksheet. Students were simultaneously able to view the video of the experimenter, see her document (i.e., case scenario, worksheet, narrative translation, checklist), observe her typing responses, and hear her vocal discussion. Because professional judgement and behavioral interpretations likely include complex covert behavior, the instructor attempted to provide an overt demonstration of any private verbal responses. For example, in the first video model analyzing sensory activities to decrease tantrum behavior, she read aloud worksheet question 6, which instructed students to distinguish between antecedent variables functioning as discriminative stimuli and those functioning as motivating operations. She discussed the antecedent variables and overtly shared an analysis of the function of those variables before typing her answer. A summary of her vocal response is as follows:

To answer question 6, I’m considering if the reinforcing value has been changed. We found in question 5 that escape is more valuable in the presence of the aversive demand and activity. Discriminative stimuli do not alter the value of a reinforcer. They signal the availability of the reinforcer, but they don’t change the value. Motivating operations do change the value of a reinforcer. That’s what’s happening here. The value of escape is being changed. Escape is more valuable in the presence of unwanted demands and activities, but not more available. So, I’m going to conclude that these antecedent variables are functioning as motivating operations.

The questions on the worksheet evoked complex intraverbal responses that often occur privately, whereas here the instructor attempted to provide an overt demonstration. After reaching a solution, the answer was recorded on the worksheet and read aloud to the students. Once the instructor answered all questions in a block, students were advised to pause the video and respond to the items in their block according to their own case scenario. They then resumed the video model, observed the experimenter responding to the next block, and rehearsed with their respective case scenario. This pattern continued until all 40 questions in all seven blocks were answered.

Following completion of the worksheet the instructor presented her narrative translation and read it aloud. Students were encouraged to again pause their video and compile their responses from the worksheet and construct a narrative translation of the nonbehavioral treatment in their scenario. The next segment of the video model showed a completed checklist where the instructor marked components as “included” or “not included” in the narrative translation. The purpose of the checklist was to minimize accidental omissions. In the video, the instructor checked the translation and confirmed that the relevant components were adequately described and marked them in the “included” column of the checklist. Items omitted were marked in the “not included” column. The instructor explained that students should confirm that any items in the “not included” column were irrelevant to the scenario and therefore intentionally omitted.

Students submitted their completed worksheet, checklist, and narrative translation via the online learning management system and the translations were scored against the scoring rubric for that specific case scenario. Corrective feedback was provided on the worksheet and/or narrative translation for each error. Praise was typically provided in the form of general statements about the translation as a whole or more specifically about responses to questions on the worksheet. Students who correctly described fewer than 80% of the essential components were given a grade of zero and instructed to review the feedback and resubmit with revisions. Once the translation was resubmitted with at least 80% of the essential components correctly described, students were given full credit for the assignment and access to the next scenario.

A total of three video models were included in the training condition. Codewords were randomly embedded throughout each video and submitted as proof of viewing the video model. After all video models were completed, the training continued with rehearsal and feedback where students were given a novel case scenario, worksheet, and feedback on their translations. Once students earned 80% or higher on the first submission for two consecutive scenarios, they entered the posttraining condition. Students who earned less than 80% for two scenarios in the posttraining condition were given a booster session that included access to the worksheet and feedback from the instructor.

Results

Figure 1 displays the percentage of essential components correctly described in the narrative translations of the nonbehavioral treatments across baseline, training, and posttraining conditions. All participants demonstrated improvements above baseline. Mean scores increased from 14.18% in the baseline condition to 75.23% in the posttraining condition.

Fig. 1.

Percentage of essential components correctly described in the narrative translations. ^aCourtney required four attempts to meet the mastery criterion on case scenario 15. Her first three translations earned a score of 43%

Courtney did not correctly describe any of the essential components in baseline and never met the mastery criterion during training to proceed to the posttraining condition. After completing scenario 18 Courtney chose not to complete the remaining scenarios. However, her translations in the training condition showed significant improvements from 0% in baseline to a mean of 72.03% in the training condition. Nick met the mastery criterion for the training condition after only five scenarios. His mean score increased from 21.56% in baseline to 79.09% in posttraining. Jackie showed similar improvements with increases from a mean score of 18.96% in the baseline condition to 74.28% in posttraining. Cody’s mean score in baseline was 16.21%. Due to an error in student access to assignments within the learning management system, Cody proceeded to the posttraining condition before meeting the mastery criterion in training. His mean score of 72.34% for the posttraining condition was slightly less than the other two participants who met the criterion in training. Nick, Jackie, and Cody all fell below the 80% criterion on two scenarios in posttraining and thus required a booster session. Nick and Cody both earned above the 80% criterion in the booster session with scores of 85.71% and 88.89% respectively. Jackie fell slightly below the criterion with a score of 77.78% correct and required revisions before proceeding.

Discussion

The current study provides an initial investigation of behavioral translations, a key component of the decision-making framework presented by Brodhead (2015). Using behavioral skills training, graduate students of applied behavior analysis were taught to translate nonbehavioral treatments into behavioral principles and procedures. Professionals from other disciplines may recommend treatments that reference hypothetical constructs, use unfamiliar jargon, and subscribe to alternate explanations for human behavior (i.e., internal causation; Brodhead, 2015). However, as Brodhead discussed, a behavioral translation of such treatments may uncover conceptual systems and indicate success in practice.

Although Nick, Jackie, and Cody all demonstrated improvements above baseline, correctness of translations fell below the 80% criterion in the posttraining phase. Removal of the worksheet in the posttraining condition may have contributed to this decrease. The questions in the worksheet may have evoked private and textual intraverbal behavior that mediated more comprehensive translations of the nonbehavioral treatment. Use of the worksheet or similar tool to identify conceptual consistencies in nonbehavioral treatments may be useful in clinical practice and further supplement decision-making models like that presented by Brodhead (2015). Thus, our goal to remove the worksheet may have been overly idealistic or premature.

The mastery criterion required participants to correctly describe 80% of the essential components identified by the expert raters. Whether or not such a criterion is necessary to fully understand the conceptual similarities between the science of behavior analysis and the nonbehavioral treatments is unknown. The opinion of expert raters was used to validate the content of the scoring rubrics for each case scenario. The components selected by the majority of expert raters was acknowledged as having some level of content validity and therefore deemed essential to a behavioral translation. Future research should seek to understand the components necessary for a comprehensive behavioral translation and, more importantly, the components necessary to inform treatment decisions in collaborative contexts. Content validation methods may include evaluations from a larger expert panel and allow components to be rated according to a 5- or 7-point Likert scale (Haynes et al., 1995). Additionally, the final translation submitted by students was not scored again by expert raters. This would be an interesting future direction. Behavior analysts could provide expert opinions regarding the correctness and comprehensiveness of the completed translations. Finally, although attempts were made to represent interventions from the published literature in other professional fields and while the primary author is credentialed in both speech-language pathology and behavior analysis, more could be done to obtain expert opinion from members of other professions regarding the chosen interventions. Future research should include expert raters from all relevant professions to confirm the procedural descriptions of the treatments.

The materials for this study included clinical vignettes, which detailed treatments from discipline-specific research literature, and fictitious client details often inspired by the participant characteristics described in the respective articles. The treatments included in the vignettes were almost exclusively published in American journals and written by authors practicing in the United States. A more culturally inclusive alternative would be to conduct database searches across geographic regions and ensure the vignettes represent diverse treatments likely to be encountered by behavior analysts working in all areas of the world. Moreover, the vignettes were written by a single author and are undoubtedly influenced by unitary experiences, beliefs, values, and practices. A broader perspective might be achieved by having the vignettes reviewed by a diverse panel of authors each representing different cultures—professional, regional, social, etc.

Participants were graduate students enrolled in an applied behavior analysis program attended virtually. Students translated the treatments described in the clinical vignettes in partial fulfillment of the requirements for a course titled Effective Collaboration Across the Disciplines. Considering the limitations imposed by course enrollment, the participant characteristics may not reflect the diverse population of behavior analytic practitioners working in collaborative settings. Ideally, recruitment measures should extend beyond a single setting and attempt to include students from diverse backgrounds. Further limitations were imposed by assignment grades. Students were given full credit for all submissions in the baseline condition. However, in posttraining, grades were based on the percentage of components correctly described in the translation. The intent was to prevent unfair penalties in the baseline condition and establish a contingency that would evoke correct responses in posttraining. It is unclear how assignment grades, given as a consequence to the translations, may have affected correctness in responses.

Translating nonbehavioral treatments is a complex verbal behavior comprised of both covert and overt responses. It necessitates a strong foundation in the science of behavior analysis and a thorough understanding of its principles and procedures. The participants all carried a minimum grade point average of 3.5 and earned a grade of B + or higher in a course covering the basic principles of applied behavior analysis, yet Courtney never met the mastery criterion in training to progress to the posttraining condition. The variability within and across participant performance may, in part, be due to differences in prior knowledge and weaknesses in the generalization of skills acquired in academic coursework. Additionally, there may be certain cultural components that contribute to successful translations of nonbehavioral treatments. That is, the unique backgrounds and experiences of individual participants may influence the breadth and correctness of translations. Future investigations may wish to examine instructional methods that promote a generalized understanding of the science, the variables that contribute to a solid behavior analytic worldview, and the prerequisite skills necessary to effectively translate nonbehavioral treatments.

The nonbehavioral treatments were based on the descriptions provided in the sources listed in Table 2. Many were noted to reference hypothetical constructs and intervening variables that students of behavior analysis may find disconcerting. For example, case scenario 14 taught completion of a vocational task using visual imagery (see Precin, 2010). A superficial analysis may find the treatment grounded in mentalism, yet a behavioral explanation would identify conditioned perceptual behavior (see Palmer, 1991) or covert discriminative stimuli which evoked the overt vocational behaviors. A more common trend observed in the nonbehavioral treatments was the use of antecedent interventions. Although not explicitly labeled as such, many of the treatments involved changes to stimuli and conditions preceding the targeted behavior. Anecdotally, participants had greater difficulty analyzing the effect of antecedent manipulations on behavior. Many behavior analytic technologies are based on the principles of operant conditioning. As such, students of behavior analysis may be taught, both in their academic program and fieldwork experience, to primarily focus on the effects of consequence changes. Future investigations may be able to identify certain behavior analytic principles and procedures that are common to nonbehavioral treatments and train accordingly (e.g., antecedent variables, private events).

The complexity of the scenarios and narrative translations made mastery and interobserver agreement difficult to obtain. It also made for a time-consuming intervention. Participants viewed a total of 218 min of video during training. Based on the available data in the learning management system, mean completion time for baseline scenarios (1–3) was 11.64 min whereas the mean completion time for posttraining scenarios (17, 18, and 20) was 52.77 min. The increased duration in posttraining is expected given the purpose of the training was to generate more comprehensive translations. The precise time needed to review, score, and provide feedback is unknown, but varied depending on clarity and correctness of translations. Future investigations should methodically track the amount of instructor time devoted to scoring scenarios and explore ways to increase efficiency for training and clinical applications. In addition, future studies might explore a narrower range of interventions and a smaller set of skills. This might help identify core skills and a more efficient instructional approach to the translational skill set.

Future directions might explore how the development of other skills could be relevant. It may be important to look at additional component skills involved in navigating team treatment decisions. Areas to explore include exposure to the published research from other disciplines, introduction to group design and statistical analyses, and instruction on the classification of interventions along the evidence-based continuum. Other ancillary skills may be equally or more important. Perhaps the greatest clinical challenges involve having difficult conversations about interventions that parents or other team members are enthused about exploring, but that pose harm or risk or are contraindicated. In these cases, the issue may be more related to interpersonal negotiation skills than to translation skills. Explorations of this area could include comparison or component analyses of such interventions and could investigate the full range of skills needed when deciding how to approach a particular intervention. Further evaluation of decision-making models for treatment selection and implementation is also needed. In particular, it might be helpful to develop decision trees that assist with paths such as exploring the procedure as a reinforcing consequence, identifying procedures that should not be pursued, and assessing the potential value of a procedure in a systematic way. Relatedly, it may be that viewing these skills through the lens of problem-solving might have merit; examining the relevance of training in systematic problem-solving or sequential decision making might be fruitful.

Conclusion

Adhering to an evidence-based model of practice may be particularly challenging for behavior analysts working in collaborative contexts. A team composed of professionals from different disciplines, each with distinct academic training, various definitions of scientific evidence, and incompatible theoretical foundations, often struggles to collaborate effectively. Each professional recommends interventions according to their discipline-specific worldview, and these perspectives must be respected. At times, these situations are problematic for the behavior analyst who must remain committed to the science of behavior analysis and its technologies all the while preserving relationships with colleagues to work in the best interest of the client (BACB, 2020; Brodhead, 2015). Some practitioners may completely disregard nonbehavioral treatment recommendations in the interest of protecting the professional relationships. They may rationalize that such treatment decisions are the product of an expertise unique to the discipline and should not be questioned. Such a position risks the application of unestablished, ineffective, or even harmful interventions. Conversely, other practitioners may scrutinize such recommendations and openly express concerns to an excessive extent, wrongfully assuming that all nonbehavioral treatments are ineffective. While such practices aim to ensure implementation of only the most effective treatments, as Brodhead (2015) noted, they can inadvertently jeopardize relationships with colleagues thus limiting the successful practice and dissemination of behavior analysis. Practitioners need tools to assess these interventions, and to assist them in deciding whether to raise concerns about the proposed intervention. We strongly believe learning to translate treatments and identify conceptual systems in nonbehavioral treatments reduces assumption-based decisions and provides a tool to expand the perspectives of behavior analysts in appreciating and respecting the work of allied disciplines. Professional judgment is critical to ethical conduct and evidence-based care, particularly in collaborative practice where nonbehavioral treatments are common. Through a behavior analytic lens, practitioners can evaluate such treatments in accordance with the science of behavior and ensure delivery of the most efficacious interventions. By learning to translate nonbehavioral treatments into behavior analytic principles, the behavior analyst comes to understand different professional cultures and potentially identify similarities that promote unity, thus helping the interprofessional team achieve a level of harmony essential to effective collaboration.

Appendix A

Sample Case Scenario: Model Scenario 1 (Modified From Bailey & Burch, 2016, p. 247)

You are a BCBA working in a public-school setting. You have been invited to join the treatment team for Cassie, a kindergarten client with autism. Although Cassie is approaching 6 years of age, her expressive language is very limited, with only a few vocal approximations to request her favorite items such as “cookie,” “juice,” “Muffy” (favorite doll), “up” (to be picked up and carried by an adult), and “Pocoyo” (favorite cartoon). She is ambulatory, but her motor problems have resulted in an unusual gait, tripping easily, and dropping things. She avoids activities other children her age enjoy, such as running, climbing, dancing, etc., and instead prefers more sedentary activities such as playing alone indoors on the floor with favorite toys or watching her favorite cartoons on her tablet. At school she usually avoids other children during free-play time and sits on the bench next to her teacher at recess.

Cassie’s teachers have expressed concerns regarding Cassie’s tantrum behaviors. When upset, she screams, drops to the floor, curls into a ball, and cries. Your functional behavior assessment indicates these behaviors are maintained by escape from difficult or aversive demands and unwanted activities.

During a treatment team meeting, Debbie, the occupational therapist states that sensory integration therapy is the best course of treatment. Debbie says, “Cassie’s scores on the Sensory Processing Measure indicate a praxis and proprioceptive dysfunction. Her hypersensitivity to environmental stimuli is causing an overreaction. Cassie needs to engage in more sensory activities so she can learn to make sense of her environment. You can see she gets on the floor in the fetal position because we are not properly challenging her senses. Whenever she is having a tantrum, we’ll need to have her participate in appropriate exercises such as rolling and bouncing on the exercise ball, playing games like tug-of-war, and jumping on the mini trampoline or accepting deep pressure stimulation such as having an exercise ball rolled over her while lying down or lying under a weighted blanket. These exercises will help her brain develop and improve her behavior. I’ll make a list of appropriate sensory activities that she can participate in whenever her tantrums start and train the classroom staff to select from the list and help her engage in these exercises.”

Appendix B

Sample Translation of Scenario Depicted in Appendix A

The overall goal of the intervention is to decrease Cassie’s tantrum behavior at school. As described, the tantrum behaviors comprise a functional response class of overt responses resulting in escape from unwanted demands and activities. These tantrum behaviors may be operationally defined as follows: an episode of crying, whining, and/or screaming following demand from adult or presentation of unwanted activity. May also include sudden drop to the floor, lying on the floor, rolling on the floor, flailing body movements, kicking, or hitting or throwing nearby objects. Does not include responses that follow painful stimuli, illness, or injury or those that may be part of a classroom activity.

The results of the functional behavior assessment indicate the tantrum behaviors are escape maintained and typically follow difficult demands and the presentation of unwanted activities. The occupational therapist has proposed an intervention that will include various sensory activities such as jumping on the trampoline, rolling, bouncing on the exercise ball, playing tug of war, etc., and deep pressure stimulation such as placing a weighted blanket over Cassie or rolling the therapy ball over her body while she’s lying down. The overt responses necessary to engage in many of these sensory activities (i.e., jumping, bouncing) are likely aversive given Cassie’s motor difficulties and preference for more sedentary activities and seem indirectly related to the goal of decreasing escape-maintained tantrum behavior.

Current antecedent stimuli, which evoke the tantrum behaviors, include difficult or aversive demands and unwanted activities. The intervention proposed by the occupational therapist does not include any changes to these aversive antecedent stimuli. These antecedent stimuli are establishing operations that increase the effectiveness or value of escape as a reinforcer and the frequency of tantrum behaviors which have been reinforced by the removal of the aversive demands/activities. Under the occupational therapist’s proposed intervention, these antecedent stimuli will be removed contingent on the tantrum behavior to allow for presentation of the sensory activities. Because the tantrum behaviors are escape maintained, this contingent removal maintains the contingency of negative reinforcement and will therefore continue to increase or maintain the tantrum behaviors. Stimulus and response prompts were not included in the description of the treatment.

The sensory activities are consequent stimuli to be presented at the occurrence of the tantrum behavior. The contingency described in the treatment procedure is as follows:

Aversive demand/activity (antecedent)—tantrum behaviors (behavior)—sensory exercises or deep pressure (consequence)

Because the sensory activities are likely aversive and contingent on the tantrum behavior, they may have a punishing effect. The school staff, teachers, and therapists implementing these procedures may be paired with these aversive activities resulting in conditioned punishers and unwanted effects of stimulus–stimulus pairing. Cassie’s preference for the more passive, deep pressure stimulation activities is unknown and the effect of these stimulus changes cannot be predicted. However, should Cassie perceive the deep pressure stimulation as pleasant it may potentially function as an additional reinforcer to strengthen the tantrum behavior.

Appendix C

Sample Block From the Worksheet

Block 1—Targeted Behavior(s)
1. What is the desired functional outcome of the proposed treatment? Is the goal of the proposed intervention a behavior or the result of certain behaviors (e.g., weight loss, having friends, earning an “A” in class)? If the goal is a behavior, is it targeted for increase, decrease, acquisition, maintenance, elimination, etc.?
2. What is the function of the behavior to be addressed by the proposed intervention? If it has not been assessed, can it be reasonably postulated based on observation or report?
3. Are there specific responses targeted for instruction with the proposed intervention? Are these responses targeted for increase, decrease, acquisition, maintenance, elimination, etc.?
4. How are the behaviors targeted for instruction related to the goal of the intervention?      a. Do the targeted behaviors represent the actual goal of the intervention (e.g., kicking a football at gradually increasing ranges with the goal to improve distance and accuracy of field goal kicks)?      b. Or are the targeted behaviors necessary for achieving the goal (e.g., increasing exercise to lose weight)?      c. Or are the targeted behaviors necessary prerequisites for another, more complex functional behavior (e.g., targeting sound–letter associations to teach reading)?      d. Or are the targeted behaviors only indirectly related to the goal? Are the targeted behaviors necessary to obtain the true goal of the intervention?
5. Does the proposed treatment recommendation target covert or overt responses? Or both?
6. Provide an operational definition for the targeted behavior(s). Can the behavior(s) be clearly and objectively defined? (Free of hypothetical constructs, explanatory fictions, assumptions, etc.)