Teaching Behavior Analysts to Address Unethical Behavior: Developing Evidence-Based Ethics Instructional Methods

Abstract

The field of behavior analysis has dictated an Ethics Code (Behavior Analyst Certification Board, 2020), requirements for instructing ethics content within a Verified Course Sequence, and the mandate that behavior analysts address unethical behavior of others. However, no evidence-based practice for how to instruct behavioral ethics, nor the specific skills for addressing unethical behavior exist. This article evaluated the effectiveness of ethics instruction developed from evidence-based instruction from other fields and applied behavior analysis’ behavior skills training (BST) with implementation of basic gamification. Comparison of student improvement on independent completion of behaviors for addressing unethical behavior were compared across applied behavior analysis (ABA) ethics classes at three different universities. The results indicated that the instructional package resulted in significantly improved student independence in addressing unethical behavior. As the first research evaluating effective methods of ABA ethics instruction, the article provides suggestions for future instructional methods and evaluation for evidence-based instruction.

To ensure behavior analysts act ethically and can differentiate ethical from unethical behavior, the Behavior Analyst Certification Board (BACB) developed a series of ethics codes. The current Ethics Code for Behavior Analysts (BACB, 2020) states that behavior analysts must behave ethically according to the Ethics Code and address ethical concerns as they arise. The Ethics Code categorizes core ethical principles (i.e., benefit others; treat others with compassion, dignity, and respect; behave with integrity; and ensure their competence) and ethical behaviors. To behave ethically, the Ethics Code states that behavior analysts must identify and define possible ethical issues, recognize the risks of these issues, identify individuals involved in the issue, address ethical issues within contextualized variables, and educate others about ethics and professional conduct.

The Ethics Code provides a decision-making process for addressing potential ethical issues and suggestions for management of code violations (e.g., obtaining data and documentation, determining and evaluating actions steps/interventions, complying with laws and licenses regulations, consulting resources). In addition to these procedural recommendations for addressing unethical situations, the Ethics Code provides several relevant, specific codes that directly and indirectly mandate that behavior analysts must attempt to intervene when observing possible ethical violations (e.g., 1.02 Conforming with Legal and Professional Requirements, 2.10 Collaborating with Colleagues, 2.19 Addressing Conditions Interfering with Service Delivery, 3.01 Responsibility to Clients). Thus, the Ethics Code provides specificity in defining and identifying ethics requirements and some general suggestions on what behavior analysts should do in correcting observed ethical violations.

Although the Ethics Code provides general suggestions for addressing unethical behavior, specific research on the instruction of ethics and on addressing others’ unethical behavior remain mostly suggestions. Behavioral ethicists suggest that applied ethics within the field should include (1) operationally defining behaviors and processes; (2) identifying complex ethical issues and principles; (3) determining possible impacts of unethical behavior from multiple viewpoints; and (4) directly addressing ethical issues (Bailey, 2021; Bailey & Burch, 2016, 2022; Brodhead, 2015; Brodhead & Higbee, 2012; Kelly et al., 2020; Romanczyk, 2017; Rosenberg & Schwartz, 2019; Schreck & Miller, 2010). However, currently, no behavior analytic research exists that empirically evaluates methods to teach students to identify and address these ethical concerns in applied settings. Given the dearth of behavior analytic research related to teaching students to identify and address ethical concerns, applied behavior analysis (ABA) instructors must rely upon research within similar human service professional and scientific fields (e.g., psychology) that has identified critical components for ethics instruction. These critical components include actions such as identification of ethical issues and relevant codes, determination of ethical solutions, and addressing concerns.

Critical Components of Ethics Instruction from Related Fields

Research in fields outside of ABA primarily has supported a variety of components for inclusion in ethics instruction classes and seminars. This research has identified effective ethics instruction to include use of (1) case-based approaches (e.g., Antes et al., 2009; Medeiros et al., 2017; Mulhearn et al., 2017; Watts et al., 2017); (2) varying instructional methods including experiential learning, practice, and skill-building throughout the course (e.g., Antes et al., 2009; Christensen et al., 2018; Medeiros et al., 2017; Mulhearn et al., 2017; Watts et al., 2017); (3) multiple learning activities (e.g., Antes et al., 2009; Watts et al., 2017); (4) group and individual activities (e.g., Medeiros et al., 2017; Watts et al., 2017); (5) prediction or forecasting of outcomes (e.g., Mulhearn et al., 2017); and (6) field specific content instruction (e.g., Mulhearn et al., 2017; Watts et al., 2017). In addition to suggesting necessary instructional components, this research has identified unsuccessful components of ethics instruction. Research has not supported exclusively academic and didactic courses (e.g., Antes et al., 2009; Watts et al., 2017), nor debates and philosophical ethical discussions (e.g., Medeiros et al., 2017; Mulhearn et al., 2017; Watts et al., 2017).

Missing Components in Ethics Instruction from Related Fields

Although research on necessary components for ethics instruction exists within fields similar to ABA, the majority of the research has concentrated on identification of ethical issues (Antes et al., 2009; Medeiros et al., 2017; Steele et al., 2016; Watts et al., 2017) and cognitive decision making/reasoning (Antes et al., 2009; Medeiros et al., 2017; Ritter, 2006; Steele et al., 2016; Watts et al., 2017). This ethics instruction research did not measure changes in students’ ethical behavior. Instruction routinely has not been based on nor measured practical application of ethical behavior or addressing unethical behavior in others, rather concentrating on mere identification and reasoning related to identification of ethical behavior (Steele et al., 2016).

Guidelines for ABA Ethics Instruction

Based upon similar fields’ research and behavioral ethicists’ suggestions for comprehensive and effective instruction, behavioral ethics instruction could generally include (1) specific instructional objectives; (2) specific ethics content (e.g., ABA-specific ethics instruction and case-specific scenarios); (3) frequent feedback; and (4) varying instructional methods (e.g., experiential learning with opportunities to engage in target behaviors, multiple activities, specific behavioral targets, combinations of group and individualized activities, and prediction of outcomes). In addition to generalizing research from related fields, ABA ethics instruction should use instructional methods from evidence-based ABA techniques (independent from ethics instruction). Use of these ABA instructional techniques may provide a robust ethics class instructional framework. In particular, this robust instructional approach could include instruction of ABA-specific ethics content and varying ABA-based or similar instructional methods (e.g., behavior skills training: BST and gamification).

Ethical Scenarios

Based upon the research suggestions in related fields mentioned above, effective ABA ethics instruction should include field-specific instruction and case-specific scenarios (e.g., Antes et al., 2009; Medeiros et al., 2017; Mulhearn et al., 2017; Watts et al., 2017). ABA ethics instructors have multiple sources from which to develop scenarios. Primarily ABA ethics instructors can find field-specific instruction in the current Ethics Code (BACB, 2020). Although ABA ethics instructors could include self-constructed ethical scenarios to use for instruction, texts also exist with examples for instruction (e.g., Sush & Najdowski, 2021).

Learning Activities

Once ABA instructors identify ABA-specific content and case-specific scenarios, they should develop an ABA ethics course to include evidence-based ethics instruction, such as experiential learning, multiple activities, specific behavioral targets, combinations of group and individualized activities, and prediction of outcomes (e.g., Antes et al., 2009; Christensen et al., 2018; Medeiros et al., 2017; Mulhearn et al., 2017; Watts et al., 2017). Approaching these varying instructional methods from a behavior analytic approach requires ABA ethics instructors to operationally define observable ethical behaviors for identifying ethical problems, addressing ethical situations, preventing ethical issues, and evaluating successful ethical interventions (Brodhead & Higbee, 2012).

Instructional Approaches

To address these ethics course needs, a BST approach (e.g., Miles & Wilder, 2009) could be beneficial. Research has consistently shown BST as effective for teaching a variety of skills, such as social skills in children diagnosed with ASD (e.g., Dogan et al., 2017), football tackling (Tai & Miltenberger, 2017), school lockdown drills (Dixon & Vargo, 2017), and various job skills performed by human service workers (e.g., Parsons et al., 2012, 2013). Generalizing the components of BST to ethics instruction would involve (1) describing the specific ethics skills to be taught (e.g., citing relevant code items; making specific suggestions for remediation); (2) providing written descriptions of ethics skills (e.g., how to approach a behavior analyst who may be displaying unethical behavior, deciding actions to take); (3) practicing and demonstrating ethics skills and behavior with corresponding data on performance; and (4) providing feedback on student performance (Parsons et al., 2012, 2013). Initial research indicates that using a BST approach may be effective in teaching identification/assessment of ethical scenarios (Piazza et al., 2023).

An additional method for involving experiential learning, multiple activities, and group/individual learning could involve the incorporation of gamification. Gamification has been considered a unique learning strategy using digital (e.g., video games) and nondigital games (e.g., simulations, board games, card games) to instruct academic material (Dicheva et al., 2015; Wiggins, 2016). Although not an ABA-specific technique, the philosophical foundations of gamification involve behavioral concepts such as (1) increasing motivation (establishing operations); (2) improving learning (skill acquisition of behavioral responses, operationally defined behaviors needed to play); (3) increasing responses through reinforcement types and schedules (e.g., rewards, points, badges, fixed ratio schedules, variable ratio schedules); (4) engaging in experiential, social repetition of behaviors with constructive feedback and no failure (prompting, errorless learning, BST); (5) providing rules and limits to behaviors; and (6) illustrating visible progress toward goals and mastery (e.g., data collection and mastery levels; Alsawaier, 2017; Dicheva et al, 2015; Morford et al., 2014; Wiggins, 2016). Research suggests that gamification may be successful in education for increasing motivation and for learning behaviors, especially if games involve game fiction (e.g., a narrative or story, ethics class case scenarios), social interactions (competition or collaboration), and experiential learning over multiple time periods (Dicheva et al., 2015; Sailer & Homner, 2020). These studies suggest gamification’s effectiveness yet indicated the need for more research using rigorous experimental designs, especially within higher education settings (Sailer & Homner, 2020).

Using these varying instructional methods (e.g., BST and gamification) to teach specific, ABA ethical content and behaviors for addressing unethical behavior in others can only currently be considered hypothetically effective. Only one published ABA example of empirical reports of effectiveness of ABA ethics-specific experiential learning, operationally defined behaviors for confronting ethical situations in the workplace, and example instructional methods could be found (Piazza et al., 2023). The current study was a preliminary evaluation of a robust ABA ethics instructional approach that involved research-suggested instructional methods (e.g., didactic skill instruction, BST, and basic gamification) to teach specific ABA ethics content and strategies to address unethical behaviors in others. In particular, the aim was to teach students how to specifically approach others engaging in unethical behavior and resolve ethical issues.

Method

Participants and Settings

Participants were master’s level graduate students enrolled in ABA ethics courses at three different universities offering a master’s degree in ABA (N = 52). Informed consent was obtained from students enrolled within each of the three universities (human subjects approval was obtained from two of the universities prior to recruitment). Each program was designated as a Verified Course Sequence. Instructors for all three courses were certified at the BCBA-D level.

University #1 was a mid-sized campus of a statewide university system located in central Pennsylvania. The instructor delivered this face-to-face course over a 16-week semester. The classroom was equipped with rows of desks and chairs, oriented towards the front of the classroom. A podium with a computer and projector were at the front of the room. Eleven first-year graduate students from University #1 (female = 9, male = 2) participated in the study. For most students, the ethics course was taken within the first four courses in the degree sequence.

University #2 was in the Midwest (Kansas) and offered the class in an online synchronous format over a 16-week semester. The cohort of participants at this university consisted of 17 graduate students (female = 15; male = 2). For most students, the ethics course was the first course in the degree sequence.

University #3 was a small college in the Northeast (Massachusetts) that also offered the class in an online synchronous format over an 11-week semester. The cohort of participants at university #3 consisted of 24 female graduate students. For most students, the ethics course was the first course in the degree sequence.

Materials

Ethical Scenarios

The authors developed scenarios describing ethical violations for baseline/pretest simulation days and for post-instruction probes. See Table 1 for example scenarios. Baseline/pretest scenarios were identical across all universities and contained potential violations from multiple sections of the Ethics Code (BACB, 2020). Scenarios for the simulation day training condition presented ethical violations that corresponded to the specific Ethics Code sections (e.g., Section 1-Responsibility as a Professional) scheduled for various class sessions in which those respective Ethics Code sections were discussed. At least one ethical code scenario was developed for each Ethics Code section (Sections 1–6). The instructors developed intervention scenarios, using a similar format, for their respective classes. The content for the scenarios derived from past course materials, personal experiences, and published materials (e.g., textbooks, articles). Post-instruction scenarios included ethical violations across the Ethics Code and consisted of a combination of instructor-developed scenarios (universities #2 and #3) and real-life scenarios obtained from practicing behavior analysts (university #1).

Table 1.

Summarized examples of scenarios for simulation days

Condition	Example Scenarios
Baseline	Bill is a 59-year-old man with autism who has been having different sleep habits over the past several months. He wakes a few times a night. Luckily, he does go back to sleep and the disrupted sleep does not seem to interfere with wakefulness or energy the next day. Recently, upon waking during the night, he showers. He is independent in the shower and nondisruptive. He does not have a roommate. The BCBA on his case informs staff to refuse to let Bill take a shower because showers are before bed or upon waking only.
Intervention Examples	You decide to attend a heavily advertised workshop by a BCBA (who also advertises as a certified eclectic therapist). During the workshop, the BCBA recommends the combined use of procedures that are not empirically verified to be effective for treating behaviors and communication deficits for children with autism. These include using sensory integration, rapid prompting for communication, and supplements in coordination with ABA programming
	Your clinical agency is consulting with Mr. Famous who developed the next best treatment to sliced bread for assessing and treating (fill in with what you wish). You know that Mr. Famous publishes every year on how amazing his “sliced bread” treatment is. Your agency wants you to use the “sliced bread” treatment on every client. Your boss and Mr. Famous told you that you need to give your graphs and data to them so that the “sliced bread” treatment can be researched. You know that Mr. Famous and your boss have a lot riding on studying and publishing the “sliced bread” treatment (e.g., consulting fees, new clients, more fame).
Post-instruction Probe:	You discover that your supervisor, who is a BCBA, is having a sexual relationship with the father of one of her former clients. The relationship started prior to the client no longer receiving services and continues after discharge.
	You are working with an adult male who receives ABA services in the home setting. He is transitioning to independent living. His parents have asked you (BCBA) to require the RBT to be physically present in their son’s bedroom to listen in on his private phone calls, despite the fact that he has asked the RBT for privacy.
	An RBT approaches you (BCBA) that their supervising BCBA alters or destroys the RBT’s daily school-based data when the daily data does not support their behavior plan. The parents only see altered data. They are unaware and confused why behaviors “only occur at home and not school.” The BCBA tells the parents that their child just acts differently in school.

Response Tables

Possible responses of behavior analysts approached about unethical behavior were constructed to be used within the practice simulations based upon professional experiences with ethical confrontation. The Response to Initial Confrontation (see Table 2) and the Response to Continued Questioning (see Table 3) options each contained six response types to be enacted by the behavior analyst who engaged in ethical violations (i.e., offending behavior analyst: OBA). Each response type had a label and examples of body language, phrases, and/or response characteristics. The six response types of each table were assigned a number 1–6.

Table 2.

Behavioral responses to initial confrontation

Die Roll	Type of behavior	Body Language and Behaviors Examples	Verbal Examples
1	Avoidant	Hands on head, head looking down, looking away	Changing the subject, denying the event, reinterpreting/rephrasing to “less objectionable” behavior
2	Dismissive	Sighing, eye rolling, sitting back in seat	Not a big deal, everybody does it, not hurting anyone
3	Defensive	Arms crossed, scowl, sitting back in seat, raising voice	Who are you, the ethics police? There is research to support it. I tried it or I believe it works.
4	Nervousness	Rapid breathing, hand wringing, picking at fingers	Uhhhh, stammering, Well ... Overly apologetic, this is hurting my feelings, changing the subject
5	Bewildered or Confused	Crinkled eye brows, shrugging, hands to face, frowning, starting into space	I don’t understand what you are saying/talking about, Can you say that again so I can understand, what is the problem here?
6	Aggressive or Confrontational	Angry face, arms crossed, shaking head “no,” sneering, interrupting, raising voice, leaning forward in seat, staring-eye contact	Who made you the ethics police? Why are you picking on me? I’m sure you aren’t perfect? It’s none of your business

Table 3.

Behavioral responses to continued questioning

Die Roll	Type of Behavior or Response	Possible Examples
1	Faulty understanding of the ethics code/research support	The ethics code says that we need to collaborate with others.
		You have read/seen at other workshops multiple accounts of the effectiveness of these treatments (google, blogs, in-service trainings by your agency, etc.)
2	Bewildered	Act as if you have no idea what the CBA is talking about—you want to behave ethically but don’t understand why they think you aren’t
3	Avoidant	Avoid the conflict, be belligerent or confrontational
4	Compromise	“Well what if I did ...” instead of “...”—you can choose the level of compromise (good or bad)
5	Competing	“Well what about you.... I saw that you did....”
6	Collaborating	Discuss with the CBA what can be done

Script

The authors developed a script to correspond with the first two steps of BST, describing the target skill and providing a written description of the skill (Parsons et al., 2012). The script described eight steps to address an ethical violation or concern (see Table 4 for script). The script steps were categorized into three phrases: (1) preparation; (2) initial presentation of the problem; and (3) listening and resolution. Each phase corresponded to a script step or multiple steps. The instructors constructed the components of the script steps based upon confrontation variables described by Keith-Spiegel (2013). The script also included a column for data collection. Each step of the script had a corresponding box to allow the data collector to score independent correct responses (+), incorrect responses (-), correct prompted responses (+P), and incorrect prompted responses (-P).

Table 4.

Script for addressing an ethical issue

Phase	Behavior
Preparation	Step 1
	Say aloud prior to Step 2:
	a. I am discharging my ethical responsibilities of my profession.
	b. I am preventing damage to clients or the credibility of my field.
	c. I am helping my colleague; I believe they are not evil or unethical; I believe this might just be an honest mistake.
Initial Presentation of the Problem	Step 2
	Greet offending BA
	Step 3
	Say something pleasant and sincere about the behavior analyst, or his/her skills
	Step 4
	Initiate the conversation. Say to the person something like: “Can I ask you about something... I was wondering what you knew about X situation?”
	Step 5
	Describe the information that you have about the problem—be objective, do not blame or be critical at this point, indicate true concern for the person
	Step 6
	Ask for the behavior analyst’s response/interpretation to the information you provided. Listen carefully to response... if satisfactory answer, you are done with situation
Listening and Resolution	Step 7
	After the response/interpretation, decide if the issue is resolved or if you need to ask questions or make further statements (repeating as needed).
	a. Ask/make questions/further statements, related to the ethical concern.
	b. If you feel an ethical problem still is likely, make some suggestions
	c. If offending behavior analyst concedes ethical point, use a reflection statement...”So, I hear you are agreeing. I’m glad you said that. So, what happens next”
	d. If offending behavior analyst convinces you that there is no ethical dilemma, use a conclusion statement...”Ok I understand.”
	Step 8
	When you determine discussion is finished, end the discussion with a professional phrase, such as “I genuinely thank you for dialoging with me about this, I feel much better”; or “I think I understand your position and I am grateful you have tolerated my expression of concern”; or “Hey, thanks for taking the time to speak with me. I better understand what is going on!”

The social validity of the script was evaluated prior to the start of the study by seven experts in the field of applied behavior analysis ethics (e.g., members of the ABA Ethics Hotline, published authors on ethics). The behavioral ethics experts provided responses on a 5-point Likert scale (i.e., definitely not appropriate to definitely appropriate) for each component of the script. The mean response for social validity was 4.6 across a possible 5 questions. Behavioral ethics expert reviewers provided suggestions for condensing the original script and providing additional examples of behavior and specific wording. The behavioral ethics experts’ suggestions were summarized and applied to the final script. All suggestions were implemented.

Dependent Variable and Measurement

The number of script steps independently completed (+) was the primary dependent variable. A script step was marked as independently completed if without prompts a student engaged in a behavior as described or exemplified in the script. For example, a greeting was scored as independently completed if a student said the name of the offending behavior analyst and a commonly used salutation (e.g., “Hello Mark”). A greeting was scored as incorrect (-) if the student started to discuss the ethical concern without first acknowledging the other party. Responses did not have to occur in a prescribed order.

For university #1, data were collected during regularly scheduled class sessions by a student data collector (see procedures below for description of student participation). Alternative data collectors (i.e., a doctoral-level behavior analyst or master’s level behavior analysts) assisted in collecting baseline data. All data collectors were trained by researchers. For universities #2 and #3, the instructors collected data by reviewing videos students posted to the course management webpage.

Procedures

Baseline and Pretest

During the initial class session, the instructors provided an overview of the project and distributed informed consent documents. The project overview was scripted. The instructor informed students that as part of the class, they would engage in simulations (e.g., role playing) to address ethical concerns and violations.

Following the introduction of the simulations and procedures to obtain informed consent, the instructor conducted a baseline/pretest evaluation. The instructor divided the class into groups of two to three students and assigned initial roles for each group member. The roles consisted of offending behavior analyst (i.e., OBA: the behavior analyst who engaged in unethical behavior) and confronting behavior analyst (i.e., CBA: the behavior analyst who would approach the offending behavior analyst about the ethical problem). For each group, a scenario was randomly assigned, among three available options, using the roll of die. The CBA and OBA were presented with a scenario corresponding to the die roll. After presenting the scenarios, the researchers rolled the dice to determine OBA response types (see Tables 2 and 3) to provide differences and randomization in responses across confrontations. The CBA was then instructed to address the ethical concern with the OBA.

No more than 15 min were allowed for completion of one scenario. No prompting, feedback, or error correction was delivered by researchers or instructors during baseline. In the event a student asked a question, the researcher said something to the effect of, “Try to do your best” or “I can’t answer that question right now.” The OBA and CBA switched roles until all students had the opportunity to participate as the CBA. Data were collected by instructors and trained master’s level behavior analysts.

Course Instruction

In the course meetings directly after baseline, students received instruction related to the Ethics Code for Behavior Analysts (BACB, 2020), information related to ethical behavior, and how to address potential ethical dilemmas that might occur in professional work. For each university course, instructors provided didactic instruction on foundations, definitions, and applications of ethics across each of the sections of the Ethics Code in numerical order (i.e., Section 1, Section 2, etc.). During class, when not engaging in simulations or data collection, the instructor conducted their class as usual. This included engaging in class discussions, reviewing ethical codes, and evaluating the applications of ethics codes to real-world situations.

Stimulation Days and Training

To instruct how to address unethical behaviors in others, the university instructors conducted simulation day practices. Prior to engaging in the simulation sessions, the instructors conducted training using BST (Parsons et al., 2012, 2013) on how to implement the script. This training occurred at different points in the semester for each university, but generally began on the second or third week of the semester and continued over a few weeks. At first, the instructors explained the rationale for the script (e.g., importance of knowing how to intervene in unethical situations, how to prevent potential harm to clients). Then, the instructors described each of the script steps involved in addressing a behavior analyst or administrator who may be engaging in unethical behavior. The third step in the training consisted of the instructor role-playing a scenario while modeling/explaining the script steps. Practicing the script steps occurred through direct practice within multiple practice opportunities across the courses. Both students and instructors provided verbal and visual feedback (e.g., review of script data) on how well the student adhered to the script.

After didactic instruction of Ethics Code sections (e.g., Section 1: Responsibility as a Professional), the instructors conducted simulation days during selected class periods. During simulations days, students practiced addressing various ethical violations. Within sections of the Ethics Code with significant information, more than one simulation day may have been conducted (e.g., Sections 2 and 3). Within some simulation days, students had multiple choices of ethical scenarios (see Table 1 for examples).

Before each simulation day, students were provided with the scenario(s), the script, and preparation tables. Prior to the simulation day, students had preparation work specific to the assigned scenario that included (1) identifying the relevant Ethics Code (BACB, 2020) to the scenario; (2) determining the seriousness of the violation; (3) evaluating stakeholders/players; (4) developing contingency plans for escalation of confrontation (e.g., next steps if the confrontation was not resolved); (5) determining professional skills required to engage in the confrontation; (6) elucidating the risk of engaging in confrontation; and (7) developing a personalized task analysis related to each of the possible OBA reactions (see Tables 2 and 3). Portions of these preparation steps were developed and expanded upon from published suggestions (e.g., Bailey & Burch, 2016, 2022).

On the day of the simulation, students grouped into triads or pairs that were rotated for each simulation. Within each triad, students served as an OBA, a CBA, and a data collector. Within pairs, students served as an OBA or CBA and the instructor served as data collector. For the in-person class, simulations were completed in a standard classroom. For the online classes, simulations were conducted in break out rooms or videotaped and submitted. Before starting the simulation, each student reviewed the scenario. Before the CBA began the initial presentation of the scenario and the script (see Table 4), the OBA rolled a die to determine the response type during the Initial Presentation of the Problem (see Table 2). The OBA also rolled the die to determine their responses to the Listener/Resolution section of the scenario (see Table 3).

Once a simulation ended, the students in the triads or pairs provided verbal and visual feedback, discussing to what extent the CBA followed the script. The OBA, data collector, or instructor offered positive and constructive verbal feedback on the quality of the CBA’s implementation of the script. After completion of the scenario, students wrote permanent product summaries of the confrontation (e.g., a progress note summarizing the confrontation, an email to the OBA, or a “next steps” plan for an unresolved scenario). Students collected data for varying numbers of training simulations across the three universities, ranging from none (university #3) to six (university #1).

Post-Instruction Probe

A post-instruction probe was conducted toward the end of the semester to evaluate the effectiveness of instruction and simulation training. For the post-instruction probes, novel scenarios were obtained from instructors and practicing behavior analysts that represented possible ethical violations across the Ethics Code sections (BACB, 2020). These scenarios provided real-life dilemmas that were not specifically assigned to a specific section of the Ethics Code. This required students to prepare and identify appropriate ethics codes across the entirety of the Ethics Code. In addition, the post-instruction probes had no feedback or prompting. During the post-instruction probe, students completed all procedures related to preparation, simulation, and completion as were completed during the training simulation days.

Experimental Design

The effect of the simulation training was evaluated using two methods. First, a pre-/post-test was constructed to allow for statistical comparison across the three university classes. This analysis allowed for statistical comparison across pre- and post-instruction. In addition, a multiple baseline across group designs (e.g., Perone & Hursh, 2013) was used to provide a more traditional within-subjects analysis across universities for pretest/baseline, simulation training, and post-instruction probes.

Interobserver Agreement

Interobserver agreement (IOA) was evaluated for 60% of all simulations (university #1), 33% of all simulations (university #2), and 33% of all simulations (university #3). IOA was measured by the instructors and graduate students within the class. IOA data collectors independently scored the number of script steps independently completed by CBAs during the simulations. IOA coefficients were calculated using total count of independently completed script steps. The smaller number of independently completed (+) script steps was divided by the larger number of independently completed (+) script steps and multiplied by 100. The average IOA for university #1 was 96% (range = 86%–100%), university #2 was 94.5% (range = 75%–100%), university #3 was 95% (range = 75%–100%).

Data Analysis and Results

Although behavior analysts often consider statistical analysis as unique in behavioral research (e.g., Fisher & Lerman, 2014), it provides appropriate support for evaluating a group-level analysis comparing pre- and post-tests across the three universities. To determine if the ABA ethics instruction resulted in statistically significant improvement in student responses, a paired samples t-test was conducted. This test evaluated the difference between students’ initial independent completion of the script steps to their independent completion of the script steps at the conclusion of the ABA ethics classes. The results indicated a significant difference between the independently correct responses to the script steps before training (M = 3.04; SD = 1.699) and the independently correct responses to the script steps after training (M = 7; SD = 1.398); [t(45) = 12.731, p < .001]. The 95% confidence interval of the difference between the means ranged from 4.58 to 3.33 and indicated a difference between the means of the samples. This statistical analysis indicated that students’ improvement in their independently correct application of script steps was due to the instruction and not due to chance.

Analyzing student improvement in completion of the script steps also was conducted across the different phases of the multiple baseline across groups. A visual analysis of the pre-and post-test comparisons from baseline/pretest to the post-instruction probe/post-test supported the statistical analysis of the effectiveness of the ABA ethics instruction (see Figs. 1 and 2). Across the three universities, students’ independent completion increased on average from 3 correct independent script steps at baseline to 7.25 independent correct script responses at the post-instruction probe phase. This indicated that students’ incorrect responses on average were fewer than .75 of the 8 script steps during the post-instruction probe. See Fig. 1 for the number of independent script steps completed by each university ethics class across baseline, training, and the post-instruction probe. In addition, across each university, individual student’s baseline responses were more distributed across the number of steps correctly completed than within the post-instruction probes. During the post-instruction probes, individual student’s responses clustered more toward the higher ends of the number of correct steps. Figure 2 shows clustering of individual student responses for each university during baseline and post-instruction probes.

Fig. 1.

Average number of script steps completed correctly at each university

Fig. 2.

Average Number of Steps and Individual Scores. Note. The bar graphs show the average number of steps completed for baseline (grey bar) and the post instruction probe (“Probe”; white bar). Individual scores across settings and conditions are depicted with closed black circles

Visual analysis of the correct independent script steps during simulation day training for university #1 and university #2 indicated a level change from baseline to training. Students at university #1 correctly completed 2 of the 8 script steps correctly during baseline/pretest, on average. During simulation day training, the number of independently correct script steps increased and remained at steadily high levels throughout the training condition (M correct = 7.3 of the 8 script steps). During the post-instruction probe condition, students at university #1 completed 7.6 of the 8 script steps, on average.

Students at university #2 completed four of the eight script steps correctly during baseline/pretest, on average. During simulation day training, the number of independently correct script steps completed immediately increased and remained at steady, high levels throughout the training condition (M correct = 7.5 of the 8 script steps). During the post-instruction probe condition, university #2 students completed an average of 7.5 of the 8 script steps.

Students at university #3 completed three of the eight script steps correctly during baseline/pretest, on average. Following training, the number of correct script steps completed increased to 6.5 during the post-instruction probe condition, on average.

Discussion

The purpose of this research was to evaluate the effects of an instructional intervention package on the behavior of addressing ethical concerns in ABA graduate-level ethics courses. The instructional intervention package included standard course instruction (e.g., didactic presentations), BST, simple gamification, and simulated practice experiences. Students and instructors from three different universities participated in the study.

Results were determined through both statistical and visual analysis and showed that all three groups of students markedly improved in implementing the steps of the script designed to approach a behavior analyst or administrator behaving unethically. All three university cohorts increased their independent correct script responses (i.e., defined behaviors for addressing unethical behavior) from pre-instruction (baseline) to the end of the ethics class (post-instruction probe). Based on visual analysis of the phases including baseline, training, and the post-instruction test probe, multiple instructional training sessions with data (i.e., university #1 and university #2) resulted in higher average levels of independent correct responses on the script steps during the post-instruction probe. However, even with simulations without data (i.e., data not collected by university #3), significant increases in independent correct responses improved from baseline to post-ethics instruction.

This study provides one of the first examinations of the effectiveness of a research-supported (e.g., inclusion of ABA-specific ethics instruction, BST, and gamification) ethics instruction approach. This study supported ABA ethics instruction that included (1) specific instructional objectives; (2) specific ethics content (e.g., field specific ethics instruction and case-specific scenarios); (3) frequent feedback; and (4) varying instructional methods (e.g., experiential learning, multiple activities, specific behavioral targets, combinations of group and individualized activities, prediction of outcomes) resulted in increased correct responses for addressing unethical behavior. The inclusion of these instructional methods within ABA ethics instruction provided evidential support to (1) evidence-based ethics instruction research from other fields (Antes et al., 2009; Christensen et al., 2018; Medeiros et al., 2017; Mulhearn et al., 2017; Watts et al., 2017); (2) ABA techniques generalized to ethics, such as BST (Dixon & Vargo, 2017; Dogan et al., 2017; Miles & Wilder, 2009; Parsons et al., 2013; Tai & Miltenberger, 2017); and (3) the effectiveness of gamification in ethics instruction (Alsawaier, 2017; Dicheva et al, 2015; Morford et al., 2014; Sailer & Homner, 2020; Wiggins, 2016).

In addition, this study provided some of the first ABA-based research to address the instruction of an ethics skill set typically neglected in ethics instruction, that of specific vocal-verbal behavior when approaching a person who might be engaging in a variety of unethical practices (Steele et al., 2016). The training script was developed to fulfill the suggestions from behavioral ethicists for what applied ethics instruction must include: (1) operationally defined behaviors and processes; (2) identification of complex ethical issues and principles; (3) determining possible impacts of unethical behavior from multiple viewpoints; and (4) directly addressing ethical issues (Bailey, 2021; Bailey & Burch, 2016, 2022; Brodhead, 2015; Brodhead & Higbee, 2012; Kelly et al., 2020; Romanczyk, 2017; Rosenberg & Schwartz, 2019; Schreck & Miller, 2010). Over the course of the study, students’ application of the script steps significantly improved, which provided preliminary evidence-based support for these applied ethics instruction suggestions.

Although this study appeared to provide the first empirical investigation into ABA-based ethics instruction, the results remain preliminary due to several potential limitations. For example, conducting experimental research across multiple universities can be a significant challenge due to instructional formats (e.g., face to face, hybrid, synchronous online instruction), semester schedules, and implementation of design. Thus, the inherent differences among the universities limit the extent to which direct comparison can be made. This limitation is inherent in all multisite research. On the other hand, student performance improved across each university despite these setting differences, which increases the external validity of the findings. Future studies investigating the effectiveness of ethics instructional methods should evaluate the possible impact of these instructional variables, tighter simulation design, validated and consistent ethical scenarios across universities, and replication of these results.

Another possible design limitation involved the use of a single baseline (pretest). As a result of only conducting a single baseline at each university, a pattern of baseline performance observed over multiple response opportunities (i.e., script responses across multiple scenarios) was not captured. Given the purpose and structure of university classes, repeated baseline without course instruction was not possible nor in the students' best interest. Class instruction (i.e., intervention related to ethics and script performance) began in each ethics class following the first session. However, because student performance would unlikely improve across multiple baseline measures because the script and procedures to address an ethical violation were not yet part of the student's learning history, this may not be a significant limitation.

In addition to limitations related to baseline measurements, it remains possible that repeated testing could have posed a threat to internal validity. The sample size for this study remained relatively small and analysis of demographic characteristics unable to be conducted. However, the sample size was sufficient for statistical analysis. Including the statistical analysis (i.e., paired sample t-test) increased the validity of the finding and reduced the impact of the single-opportunity baseline and repeated testing. However, a limitation remains that some student improvement could be accounted for due to the repeated testing/practice.

The effectiveness of the ABA ethics training from baseline to post-instruction probe could only be determined based upon the combination of the instructional methods (e.g., BST + gamification + didactic instruction). The results of this study were obtained by analyzing the totality of the instructional method. The results provided some support for the effectiveness of the instructional method, but significant additional research must be conducted to evaluate which instructional components resulted in the most behavioral change, if additional script steps may be needed, or if other instructional methods should be added. For example, the gamification used in this study involved a molar level of gamification (e.g., dice rolling for responses, immediately earned points for correct responses, simulations). Improvement could be directly related to the rehearsal components within BST and the dice rolling directly related to generalization of responses. Future research could evaluate the effectiveness of additional components of gamification, such as leaderboards/public data, competition/cooperation, increased game rules and strategies, implementation of more strategies and reinforcement schedules (Morford et al., 2014).

In addition to the evaluation of the instructional components, future research could evaluate the generalization of script use outside of the classroom setting. For this study, the post-instruction probe provided an attempt at generalization for scenarios across all possible Ethics Code sections (BACB, 2020) and across real-life examples. However, no direct measure was collected to determine if student performance in a class simulation resulted in generalization to responding to unethical behavior in the real-world. Although perhaps difficult to do, this real-world application would be tremendously useful to assess.

Although this study’s limitations may limit the generalizability of the instruction across ABA ethics instruction, the results provided some of the first empirical support for evidence-based instructional methods for ethical applications. The script used to teach these ethical applications comprehensively covered ethical behavior and conduct, such as (1) using evidence-based practice; (2) identifying ethical issues and risk; (3) addressing ethical issues within contextualized variables; (4) educating others about ethics and professional conduct; and (5) documenting and evaluating actions (BACB, 2020). The methods described in this study could be generalized, replicated, and expanded within classroom instruction and supervised fieldwork to progress the field toward identifying evidence-based ethics instruction. Implementation and expansion of these instructional methods also could result in a reduction of behavior analysts reported for ethical violations, and, most importantly, an increase in behavior analysts behaving ethically.

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Declarations

Ethical approval

Approval was obtained from the ethics committee of Pennsylvania State University-Harrisburg and University of Kansas. The procedures used in this study adhere to the tenets of the Declaration of Helsinki. Informed consent was obtained from all individual participants included in the study.

Competing interests

No funds, grants, or other support was received. The authors have no competing interests to declare that are relevant to the content of this article.