Curriculum-based Evaluation of Cultural Competency Coursework in an Online Applied Behavior Analysis Graduate Program

Abstract

Culture affects the practice of applied behavior analysis in many ways. An individual’s family culture or a community’s culture often influences what behaviors are considered appropriate or inappropriate. Practitioners of applied behavior analysis need to adjust service delivery to accommodate the unique language and cultural needs of each of our clients to deliver behavior-change interventions that are likely to be accepted and adhered to after the behavior analyst is not present. It is important that this focus starts while practitioners are developing their own practice behaviors, as early as when they are students. This study explores a method for evaluating the effectiveness of cultural competence coursework using embedded assignment questions based on complex case examples. A quasi-experimental design was used to evaluate the effects of cultural competence coursework on increasing student knowledge of how culture can affect human behavior. In a course on functional behavior assessment, responses to specific scenarios, including cultural and disparity concerns, were provided to students. The responses of the students who completed a cultural competence course prior to taking the FBA course were analyzed and compared to those students who had not yet taken the cultural competence course. Results showed that students who had previously completed the cultural competency course were more likely to identify cultural variables only after completing a subject-specific training module in the FBA course.

In his seminal paper on social validity, Montrose Wolf recounted his initial trepidation at suggesting “social importance” as a guiding principle for the field due to the difficulty in measuring what is important to society through objective, measurable behavior change (Wolf, 1978). Social validity was established to solicit important information from the people and communities most greatly impacted by the work of behavior analysts. Although the term “culture” is not explicitly defined or even used, Wolf’s goals of being responsive to the needs of communities and creating conditions in which consumers of behavior analytic are presumed the most qualified to evaluate the effects of the services they received were clearly a recognition that individual and community differences can impact the acceptability and effectiveness of applied behavior analysis (ABA) services (Wolf, 1978, p. 212).

This line of thinking was continued in 1987 when the original framers of ABA revisited the seven dimensions they defined in 1968. It was concluded that larger issues of social validity, and thus cultural relevance for the consumers of the science, can be addressed using the established science of ABA. This type of focus on the validity of the application of the science, however, needed to be expanded and built upon to ensure an effective fit for the consumers of behavioral science (Baer et al., 1968, 1987). The contributions of the pioneers of the science of behavior continue to be relevant today in a discussion of how culture affects the practice of ABA in all settings where it is practiced (e.g., homes, communities, schools) worldwide.

Despite the foundational elements of incorporating social validity into the practice of ABA, our field has been remiss in adopting a method for using the technologies of ABA to address the needs of culturally diverse populations (Zarcone et al., 2019). Organized first steps toward this goal were not taken in earnest until the Culture and Diversity Special Interest Group (SIG), composed of behavior analysts dedicated to exploring the need for diversity in the field of ABA and supporting the efforts of behavior analysts working with or members of historically marginalized groups, established the Standards for Cultural Competence in Behavior Analysis (Fong & Tanaka, 2013). Fong and Tanaka (2013) described culture in terms of the way different groups experience the world around them and pointed out that no one is without culture and that even the most highly trained scientist is affected by their cultural perspective. This contribution to the field led to a sizeable increase in the discussion of culture within the field of ABA. Key terms related to culture, practice, and training continued to be defined and are still evolving.

The concept of cultural awareness has been defined as understanding one’s own culture and seeking to understand the culture of others (Fong et al., 2016). Cultural awareness was described as a key skill for behavior analysts to effectively practice within a culturally diverse context (Fong et al., 2016). Specific strategies for practitioners developing cultural awareness of self and clients included but were not restricted to considering the language needs of their clients, the resources required to implement interventions, and the cultural identity of all the interested parties in our clinical work (Fong et al., 2016). Structural and organizational supports that would improve cultural awareness were described as including increased diverse representation in the workforce and graduate training programs so that people from all backgrounds would feel that they are welcome in the field (Fong et al., 2016).

To help identify the barriers to a diverse workforce in the field of ABA, issues of diversity, equity, inclusion, and belonging were explored through a Critical Race Theory framework (Fong et al., 2017). This exploration found both cultural awareness and cultural understanding to be important components in developing culturally competent practitioners (Fong et al., 2017). Cultural competency has been defined as behavior analysts having the necessary skills to practice effectively within the cultural contexts they find themselves (Fong et al., 2017). As our inquiry into issues of diversity and the importance of practitioners becoming fully culturally competent emerged, the term cultural competence led to a shift in focus to cultural humility (Wright, 2019). Cultural humility was defined as practitioners understanding that everyone is an expert in their own culture but not necessarily the culture of others, centering the needs of others, and committing to a life-long process of learning and responding to changing cultural conditions (Wright, 2019). Despite the call to use the term cultural humility, cultural competence may still be a relevant term if it is defined not as mastering knowledge of other cultures, which is not attainable (Wright, 2019), but rather as the process of building awareness, as well as the knowledge and skills necessary to effectively complete a task while considering cultural needs (Sue & Torino, 2005 as cited in Beaulieu & Jimenez-Gomez, 2022). For example, if the larger, programmatic goal for a training program in behavior analysis is to produce culturally humble practitioners, being competent in the cross-cultural application of behavioral assessment technologies within a variety of contexts is an important skill to target as a step toward that goal. This type of competence, part of the culturally responsive practice of ABA (Jimenez-Gomez & Beaulieu, 2022), remains particularly relevant for students who are not yet practicing or in the early stages of practicing and working toward attaining the skills or competencies required to be culturally responsive and culturally humble practitioners once their training is complete.

One area where the application of cultural competency in behavior-analytic work is especially needed is in the cross-cultural application of functional behavior assessment (FBA) (Jessel et al., 2025). FBAs are employed primarily to identify the factors interfering with a person functioning successfully in their environment. A person’s culture, without question, plays a significant role in the reasons for all behavior in which we engage. In fact, some behaviors that may be deemed as interfering or problematic may be directly tied to differences in the culture of the individual displaying the behavior and the person identifying the problem. Further, all components of an FBA, including the initial interview with the individual and the individual’s caretakers, the definition of behaviors to target, and the choice of goals can all be impacted by the cultural competence of the provider (Jimenez-Gomez & Beaulieu, 2022). For example, research has shown that Black male students are more likely to be suspended from school when displaying the same types of behaviors that their White counterparts display (Huang, 2020). This points to subjectivity when interpreting behavior as problematic, which is not typically accounted for in behavior-analytic training programs (Conners et al., 2019). A behavior analyst who is called upon to conduct an FBA, which includes selecting and defining behavior, needs to be aware that how a teacher or caregiver describes a behavior may not be objective or culturally responsive. Trainees receiving explicit training in the cross-cultural application of behavioral assessments will be better prepared to combat and rectify these issues (Mathur & Rodrigues, 2022).

Cultural factors in the form of setting events can also affect behavior (Beaulieu & Jimenez-Gomez, 2022). Setting events are environmental influences that are often found outside of the immediate, operant contingency (O’Neill et al., 2014). An example of the immediate operant contingency is the amount of time that has passed since food has been consumed, a well-established motivating operation that alters the effectiveness of food as a reinforcer. If an organism has not eaten recently, it will be more likely to behave in a way that leads to accessing food, or food becomes established as a reinforcer. Conversely, satiety or “being full” is the abolishing operation that decreases the likelihood of food being an effective reinforcer. These stimulus conditions are part of the immediate antecedent of an operant contingency. Food insecurity is a setting event for some people because it is a related environmental condition that may not be a part of the immediate operant contingency but may still impact behavior involving food. Food insecurity, or lack of consistent access to necessary nutrition, is more likely to be experienced by those in marginalized communities, such as people experiencing poverty (Berkowitz et al., 2024). For example, if an individual is engaging in food-seeking behavior, satiating that person’s hunger should abolish food as a reinforcer, thereby reducing the value of food and instances of food-seeking behavior. However, it may be the case that feeling full does not reduce the value of food for a person who is experiencing food insecurity. Behaviors such as food stealing and food hoarding among children in foster care may be an example of this, as it is thought to be associated with a history of neglect or hunger (Casey et al., 2012).

The need for more significant training in behavior analytic training programs to increase the cultural competencies of practitioners that, in turn, produce meaningful real-world outcomes is apparent. Behavioral scientists have made recommendations to incorporate more diversity, equity, and inclusion (DEI) readings into graduate training programs (Hollins et al., 2023; Ortiz et al., 2022), but this alone is likely not sufficient to produce meaningful behavior change. Although behavior analysts training in cultural competencies often takes place in the academic setting, there is also a need to apply what is learned to behavior analysts’ employment settings (Mathur & Rodriguez, 2022). One major shortcoming in training for cultural competency is the lack of a direct mechanism for measuring a student’s behavior change in the area targeted. To measure the efficacy of training in cultural competency, some studies have used indirect measures such as online cognitive tests (van Ryn et al., 2015) or self-report measures such as surveys and checklists (Purnell, 2016) to measure implicit biases or cultural awareness. These measures, however, are insufficient for several reasons. Identifying one’s implicit biases does not necessarily speak to that person overcoming those biases or that biases necessarily lead to discriminatory action (De Houwer, 2019). Similarly, although self-assessment and reflection are important, best practices for self-assessment have not been established, and errors such as confirmation bias may lead to practitioners only attending to the behaviors, thoughts, or information that endorses their existing beliefs as opposed to challenging them (Beaulieu & Jimenez-Gomez, 2022). Finally, indirect and self-report measures are not consistent with the direct, observational data collection behavior analysts typically use to evaluate behavior change. Despite the field’s overall focus on measuring behavior change, there is a dearth of individualized and behavioral measures of increased cultural competence among behavior analysis trainees in the educational environment, as well as research evaluating the effects of educational strategies to improve cultural competence.

Effective behavior analytic practice requires that practitioners consider the social validity of the work they do (Wolf, 1978). To do this, practitioners must consider how cultural variables can impact behavior (Jimenez-Gomez & Beaulieu, 2022). As per the central dimensions of ABA, the effectiveness of behavior change programs is based on the measurement of observable behavior (Baer et al., 1968). Unfortunately, there is often a disconnect between what a student learns in class and what is needed for compassionate and effective practice in their work settings (LeBlanc et al., 2020). This study measures the application of what they have learned in a Cultural Competency (CC) course in a subsequent FBA course as a tool for instructors to evaluate the effectiveness of the targeted cultural competency skills while remaining within the academic setting.

Method

Participants

All participants were graduate students in an online graduate program who completed a required graduate-level FBA course between February 2022 and December 2022. Each online semester was 8 weeks in length, with two semesters in the Fall, two semesters in the Spring, and one semester in the Summer. According to demographic data for the online graduate program in ABA at the time of the study, most students identified as White (82%) and female (90%). Student data analyzed for this study were collected from students enrolled in the online Functional Behavior Assessment (FBA) in the Spring 2022 and Fall 2022 semesters. The FBA course was offered twice per year. A total of 82 students were enrolled in the FBA course during this time. Of the 82 students, one student’s data was excluded due to not completing the course, two students’ data were excluded due to taking the CC course concurrently with the FBA course, and two students’ data were excluded due to completing only one of the two-course components analyzed for this study. A total of 77 students’ data were included in the analysis. Of the 77 students, 43 had not yet completed the CC course (55.8%), and 34 (44.2%) had previously completed the CC course earning a grade of B or higher (see Table 1). Of the 34 students who had previously completed the CC course, 85% had completed the CC course in the semester immediately preceding the FBA course, and 15% had completed the course when it was offered the previous academic year. At the time the master’s degree curriculum was developed, there was no content similar to the CC course in any other courses in the master’s degree program.

Table 1.

Students enrolled in the FBA course in the Spring 2022 and Fall 2022 semesters

Students	FBA course only	FBA + CC course
Participants- 77	43	34
Total enrolled- 82	–	–
Excluded- 5	–	–

Research Design

A nonexperimental, correlational design was used to determine if there was a relationship between the cultural competency coursework (i.e., the three-credit CC course and the subject-specific embedded module) and the student’s performance on the case example components. The comparison was conducted in two phases. Phase one occurred in week 2 of the FBA course, and the performance of students who had taken and did not take the CC course was compared. Phase 2 of the study was conducted by administering a subject-specific embedded module to all the students and then readministering the dependent measure to compare the performance of the same groups.

Independent Variable

The independent variable was participation versus non-participation in a one-semester CC course titled Issues of Ethnic and Cultural Diversity. This course was a 3-credit, asynchronous, online course that was required for all students graduating with a master’s degree in ABA in New York and was offered twice each year. It is important to note that although all students seeking New York State licensure at the time of this study were required to take the CC course, this course had no prerequisites in the ABA program. Therefore, it was possible to take the course at various times in a student’s program plan depending on the student’s personal needs, schedule, and other variables such as course repeats or transfer credits. Consequently, some students took the CC course prior to the FBA course, while others took it after.

Coursework Description: A Framework for Addressing Cultural Competence in Online Behavior Analysis Courses

 The development of this course was guided by the Multicultural Alliance for Behavior Analysis’ seven standards for culturally competent behavior analysis as course content (Fong & Tanaka, 2013; Multicultural SIG, 2019), included readings from a required text (Conners & Cappel, 2020), posted articles, recorded online lectures, and supplementary online videos. During the CC course, students used the posted materials and the readings to complete reflections based on case examples and completed self-awareness-building activities described in more detail below.

Proposed Standards for Cultural Competence in Behavior Analysis

As stated previously, the Multicultural Alliance for Behavior Analysis proposed seven standards of culturally competent behavior analysis (Fong & Tanaka, 2013). The seven standards of ethics and values, self-awareness, making ethical referrals, cross-cultural application of ABA technologies, accommodating language diversity, encouraging a diverse workforce, and engaging in ongoing professional development were central to the course objectives and activities of the CC course. Learning about the ethics and values of the field of ABA required students to not only be familiar with the Behavior Analyst Certification Board’s Ethical Code (BACB, 2020) but also to reflect on the value of diversity in the field of behavior analysis. Self-awareness was fostered by students reflecting on their own potential biases as well as their own cultural background and identity. By examining their own beliefs, biases, and cultural knowledge, students could then develop a process to help them identify areas that require additional education or training. Similarly, students learned about when and how to ethically refer a client to another service provider who would better serve the cultural needs of the client if the need arose.

Students reflected on applying common behavior analytic technologies with multiple populations, including those who speak a different language from themselves. Students were also required to consider how they could best develop a diverse workforce, deal with cultural conflicts, and create a plan for ongoing professional education as they became the next supervisors and employers after their education and training were complete. Many of the learning activities in the CC course required students to read relevant texts and articles and then reflect on presented case examples.

Course Activities: Analysis of Complex Case Examples

To allow students an opportunity to meaningfully understand how to practice based on the competencies outlined by Fong and Tanaka (2013), students were asked to reflect on the practice of behavior analysis with cultural groups different from their own through the pairing of case examples with relevant readings each week. To ensure the course was responsive to the culture and background of the students, an element of choice was included in the CC course. In the “choose your own adventure” modules, the entire class engaged in a behavior analytic focus topic, for example, the cross-cultural application of common behavioral assessments. The module also contained three examples of potential cross-cultural applications. Each cross-cultural choice included a reading, a first-person account in the form of an online video, and a case example. Students were required to reflect on at least one cultural group when developing their plan for the cross-cultural application of the selected ABA technology. This provided the students access to a variety of topics while also allowing them to select an experience that would be most relevant to their background, personal goals, and career path.

In addition to the standard of cross-cultural application described above, the same mechanism of complex case examples was used as a tool to facilitate student reflection in the areas of language diversity and supervision. For example, students were asked to develop items for an intake questionnaire based on a case example that included the added consideration of a multilingual family (Conners & Capell, 2020). Finally, a unique case example about a cultural conflict between a family and a supervisee was used to help students reflect on how to develop a diverse workforce by supporting a supervisee from a historically marginalized background.

FBA Course Specific Module

 In addition to the three-credit course described above, a one-hour module on culturally responsive functional behavior assessment (FBA) was also developed. The module consisted of an interactive, asynchronous, online lecture in two parts. The first part of the lecture was defining key concepts and vocabulary related to multiculturalism and diversity in the practice of ABA (Conners & Capell, 2020). This content was also presented as a part of the CC course. The second part of the lecture was specifically related to culturally responsive assessment (Jimenez-Gomez & Beulieu, 2022). The lecture was made interactive via a variety of multiple-choice and matching activities embedded throughout the presentation. Students were required to complete the activities to proceed through the lecture, ensuring active participation. All 77 students whose responses were analyzed for this study satisfactorily completed the embedded, course-specific learning module.

Dependent Variables

Student responses to two open-ended components in the FBA course served as the dependent measures for this study. The FBA course served as the test environment to evaluate the effects of the cultural competency course on subsequent performance on culturally related assessment questions.

Using the FBA course to evaluate the impact of the CC course content served two purposes. First, the FBA course was a new environment that may have indicated application of what they learned in the CC course to the professional practice of conducting functional assessments. The second benefit was that the course was a part of the student’s educational program at the time of the study. This meant that if students performed poorly on the culturally related elements, indicating a lack of effectiveness of the CC course, faculty could respond by adding content to one or both courses in the future if needed.

Embedded Components

Students’ responses on two selected case studies that included four contextual factors or setting events that were completed in the FBA course were used as the dependent variables. Students were asked to write an interview question about one setting event from the case study. One of the setting events embedded in each case study was related to an aspect of culture (socio-economic status and religious/cultural dietary restrictions). The case studies were developed by the instructor based on cultural variables that had previously been covered in the CC course. Both case studies were based on children diagnosed with autism because most students worked in this area while they took part in the online master’s program.

During the course, students received credit for their correct identification of any of the four potential setting events (i.e., food insecurity was a culturally relevant setting event while medication effects were not) and their creation of a question regarding that setting event to include in a Functional Assessment Interview (FAI) irrespective of the cultural relevance of the setting event they identified. The grade the students earned was not related to identifying the cultural variable. Each case study was worth a total of five points. Students scored, on average, 4.3 points out of the possible 5 points with scores ranging from 1/5 to 5/5 across both assignments, 86.5%. Grades on the setting event identification questions were similar regardless of the student’s CC course status.

Between the two case study activities, a course-specific module on culturally responsive behavioral assessment was administered.

Case Study 1

Was completed in week two of the eight-week FBA course as a part of the curriculum content on indirect measures of behavioral assessment such as the Functional Assessment Interview (FAI) (O’Neill et al., 2014). The assignment involved developing an interview question for a specific case example as part of the initial FBA process.

Case Study 2

Was completed at the end of the 8-week semester. A question similar to the selected item from the FAI assignment was added to the final exam of the FBA course. Both case studies can be found in Appendix A.

Data Coding

Students’ responses were first coded to identify which of the four embedded setting events students selected. If they did not correctly identify a setting event, the response was coded as incorrect. Student responses were then analyzed and scored depending on whether the cultural factor was the setting event they chose to write about (scored 1). If one of the other setting events was selected, it was scored 0.

Data Analysis

A chi-square test for independence was conducted to examine the relationship between completion of the CC course and performance on the case study activities. Results of a chi-square test for independence indicate whether there is an association between two variables. In this case, the analysis was performed to determine if students who had completed the CC course were more likely to identify the cultural setting event. This nonparametric statistical test was selected due to the categorical nature of the data.

Institutional Review Board

Because only deidentified, existing student data were used in the study for the purposes of course evaluation, Certificates of Exemption were obtained from Internal Review Boards (IRBs) of both the university the students attended and the university the author attended at the time of the study.

Interobserver Agreement (IOA)

Interobserver agreement was used as a measure of reliability for the analysis of student responses to the selected questions based on the complex case examples. Second raters consisting of master’s and doctoral-level behavior analysts coded 30% of the student responses to indicate which of the possible setting events the student wrote about. To determine agreement, the primary rater’s code of each student’s response was compared to the code provided by the secondary rater. Possible codes for each student response were any of the four embedded setting events or “incorrect.” Coding students’ written responses to open-ended questions presented a unique challenge as there was more than one possible code for each student response. Because of this, both total agreement and exact agreement were calculated.

For exact agreement, an agreement was defined as the secondary rater identifying the same setting events as the primary rater. Partial agreements were scored as a percentage. For example, if the primary rater coded a student response as identifying schedule changes as the identified setting event and the secondary rater coded schedule changes and medication effects as identified setting events, it would result in a 50% agreement for that student response. Owing to the high number of partial agreements and the open-ended nature of the student responses, exact agreement was calculated to be 69% of the analyzed student responses. For total agreement, an agreement was defined as the secondary rater identifying the same setting event as the primary rater. Additional setting events identified by the secondary rater were disregarded. Because the student responses were ultimately scored on a binary basis, regardless of whether the cultural variable was written about, an exact agreement is not necessary. The total agreement was calculated to be 91%.

Results

At the first assessment point in the FBA course (week 2), there was no difference in writing about cultural variables between the students who had (11.7%) and those who had not (11.6%) previously completed the cultural competencies course. However, after the students in the FBA course completed the embedded subject-specific module on culturally responsive functional behavior assessment, students who had previously completed the cultural competency course (67.6%) were significantly more likely to write about the cultural variable on the final exam compared to students who had not yet taken the cultural competency course (39.5%) (see Fig. 1).

Fig. 1.

Student performance on selected course items. Note. Module on culturally responsive FBA was delivered in week 8; Final exam was at the end of the 8-week semester

A chi-square test for independence was conducted to examine the difference between the performances of students who had and had not previously completed the CC course. Statistics were calculated with the assistance of the data analysis service Julius AI (Julius AI, 2024). There was no difference between the two groups in the week 2 assessment. However, the results of the chi-square indicated an association between CC course status and performance on the final exam question, Chi-square (1, N = 77) = 5.35, p = 0.021 (see Table 2). These results indicate that the CC course alone did not impact student performance but that students who experienced both the CC course and the course-specific module were more likely to identify the cultural variable in the case examples than their classmates who experienced the course-specific module alone.

Table 2.

Cultural competency course status and identification of cultural variable embedded within selected components

Component	FBA course only		FBA + CC course		p-value
	n	%	n	%
Week 2	5	11.6	4	11.7	1
Final exam	17	39.5	23	67.6	.021*

^*A significant relationship between cultural competency course status and performance on the final exam item was found

Discussion

The results of this study support the use of subsequent coursework to evaluate both measurable behavior change and generalization resulting from the completion of a cultural competency course. Specifically, this study shows that within an online professional graduate program in behavior analysis, instructors can measure the effects of specific coursework and take steps to encourage the application of earlier coursework by embedding activities that target skills introduced earlier in the curriculum. The results of this study also indicate that when subject-specific stimuli are introduced after the initial training or initial coursework has been completed, students perform significantly better than their classmates who had not completed the previous coursework, even when initial generalization of the previous course content was not demonstrated. This is consistent with what we know about the limitation of a “train and hope” strategy for generalization and programming common stimuli to increase the likelihood of generalization to relevant settings and situations (Stokes & Baer, 1977). In this way, the subject-specific module can be seen as an extension of the initial CC coursework rather than a test for generalization. It is not until the students experience both the CC course and the subject-specific module that we see a significant increase in the identification of the cultural variables.

These results also have implications for future coursework in the field of ABA. At the time of this study, the two regulatory bodies governing the licensure and certification of behavior analysts in New York State had different recommendations and regulations regarding how best to instruct behavior analysts in the culturally responsive practice of ABA. NYS regulations required that master’s degree programs leading to licensure in ABA included a 3-credit stand-alone course in “issues of cultural and ethnic diversity.” In contrast, recent research has explored the effects of incorporating learning objectives in Culturally Responsive Service Delivery (CRSD) throughout graduate programs in ABA (Ortiz et al., 2022).The Association for Behavior International (ABAI) higher education accreditation standards that will soon be required provide no specific guidance on delivering coursework dedicated to the culturally responsive practice of ABA (ABAI, 2015). On the basis of the results of this study, students were most likely to recognize and choose to write about potential cultural variables after they had experienced both a stand-alone course and embedded, topic-specific content. Since this study was conducted, the NY State Board of Education removed the stand-alone cultural and ethnic diversity course requirement. However, the results of this study indicate it may be very beneficial for programs to continue to include a course with a specific focus on cultural competency as part of their curriculum.

This study had several limitations and considerations when interpreting the results. First, the second administration of the independent variable (the content-specific module) was temporally close (within a week) to the second administration of the dependent variable in the final exam. Although a significant between-group difference was still noted, the increase in students choosing to write about the cultural variable is likely due, in part, to the relatively recent exposure to the content-specific module when compared to the stand-alone course taken in previous semesters.

Two other limitations of this study are the correlational design and the lack of random assignment. Because students were not randomly assigned to certain conditions, we cannot say for certain that there was no other variable that could have accounted for the differences in responses. For example, if students took other courses or training related to cultural competency that were unknown to the experimenter, this could have influenced the results. Despite the lack of random assignment, it is unlikely that more students who had previously taken the CC course also completed additional training; however, it is possible. Students may be exposed to content related to culture and diversity through their employers or even popular media, and an overall increase in students’ sensitivity to cultural considerations may be expected with time.

Although the correlational design is an experimental limitation, it is also, logistically, an advantage due to the ease of implementation for educators. Similarly, this study does not use a control group or a true baseline measure for all participants. For increased experimental rigor, the dependent measure could be administered to a group that had not received any cultural competency training throughout the study to serve as a comparison group for those students who completed the cultural competency module as part of the FBA course. Again, although using a control group would be experimentally more rigorous, it would not be in the best interest of the students to miss or delay important educational content. One other possibility would be to conduct assessments immediately prior to taking the CC competency course to conduct a multiple probe design. However, this could also be a confounding variable as multiple exposures to the dependent measure alone may impact student responses.

Although a statistically significant difference was found in the final exam question, expanding this study to include more students would allow for the exploration of interactive effects between the CC course and the subject-specific module. A better understanding of the way the two training components interacted could provide valuable information for educators. The difference could be accounted for by a “dosage effect,” meaning that there is a certain level or intensity of training required before an effect is seen, or it could be that periodic exposure to the instructional material encouraged the maintenance of the skills acquired in the CC course.

This study is a first step in curriculum-based evaluation of coursework in ABA training programs. As the field of ABA continues to call for diversity, equity, and inclusion content to be a part of graduate curricula in ABA (Hollins et al., 2023), educators can build upon this framework to evaluate the effects of CC content within ethics courses or courses focusing on organizational behavior management. Similarly, educators can use different outcome measures in lieu of complex case examples. For example, using simulation, role play, or even supervisor observations as a measurement of generalization may further enrich the quality of cultural competency training in the field of ABA, both in the education of students and the continuing education of practicing behavior analysts (Mathur & Rodriguez, 2022).

The current line of research represented by this study should continue to be evaluated, and efforts to improve the efficacy of education and training in ABA should be explored further. Specifically, by using outcome measures such as simulation or performance on complex case examples, the efficacy of standalone CC coursework should be compared to embedded topics in existing courses. Moreover, future research should try to determine the amount of training sufficient to produce the desired outcome of culturally responsive and humble practitioners. Owing to the complexity of these questions and the many ways CC coursework can be implemented, coordination and communication among supervisors, instructors, students, and consumers of behavior analytic services should be encouraged when developing any mechanism for evaluation of educational content for behavior analysis students. This study advances the possibility of the behavioral measurement of skills related to the culturally responsive and culturally humble practice of ABA that are challenging for educators to define and measure. Continuing this line of research will help our field develop objective, measurable, and valid interventions to positively impact the practice of behavior analysis and the delivery of ABA services.

Appendix A

Complex Case Studies

Case Study 1 presented early in the semester in the FBA course.

Mari is a 4-year-old girl diagnosed with autism spectrum disorder living in the Washington Heights neighborhood of Manhattan. She has been referred to Any Behavior Agency, where you work as a behavior analyst, by her pediatrician for assistance with severe challenging behavior she is experiencing at home, in particular on the weekends. Mari is participating in an Autism Waiver Program through Medicaid designed to help families access support systems for their children with autism. Through this program, an initial review of the school records and a telephone screen with Mari’s primary caregiver have been completed.

School record review summary: Mari is currently attending Washington Heights Education Center and is in a 6:1:1 classroom (6 students, 1 teacher and 1 teacher aide). According to her educational record and teacher questionnaire, Mari has not displayed challenging behaviors in the past 6 months at school and what challenges did occur at the beginning of the year were thought to be a result of being in a new classroom with new routines and resolved quickly with minimal intervention. Mari receives breakfast and lunch through an assistance program at school and is often very hungry when she arrives in the morning. She responds well to predictability and routine. She doesn’t like it when her schedule is changed, and she enjoys reviewing her picture schedule multiple times per day.

Intake information: Mari currently lives with her mother and 3 older siblings. She shares a room with her older sister (6) and her two brothers (8 & 10) share the room next door. Mari’s mother works long hours during the week and every other weekend as a paralegal assistant in a nearby law office. She is currently finishing her degree in paralegal studies online in the evenings. She relies heavily on her mother (Mari’s grandmother) for assistance with childcare.

Recently, Mari has been struggling when her mom is working on the weekends. Her grandmother reports that Mari refuses to get out of bed most mornings and once she does get out of bed, she is frequently aggressive toward her siblings. She is non-compliant and unpredictable most of the weekend with her grandmother and will hit and kick her brothers and sister if they have a toy or snack that she wants. After lunch, she takes a nap and is usually in a much better mood when she wakes up.

Students were asked to complete several questions based on the case example. The dependent variable was specifically the student generated answer to the following 2-part, open ended question:

Are there any larger, contextual characteristics that you feel are relevant to Mari’s problem behavior? Please write 1 sentence explaining what ecological or contextual factors may be impacting Mari’s behavior (pick 1 or 2). Write 1 follow-up question that would help you get more information about this issue (or issues).

• Students were asked to identify general ecological or setting events that influence the sensitivity of the problem behaviors to the maintaining contingencies and then craft an interview question designed to obtain additional information about their chosen setting event. Identifying any of the four potential setting events and writing a question about it were considered correct responses for the FBA course.

• The following ecological or setting events were part of the scenario.

  1. change of schedule
  2. change of caregiver
  3. sleep concerns
  4. poverty/food insecurity (cultural variable)

• For the purposes of course evaluation, student responses were re-analyzed to determine how many students identified and wrote a question about poverty/food insecurity as a setting event. Low socio-economic status is a topic specifically targeted in the cultural competency course. Identifying this setting event may indicate that the content of the cultural competency has generalized and that students are applying what they learned to their analysis of case examples in subsequent course work.

Case Study 2 presented as part of the final exam in the FBA course.

Throughout the course of the semester, the students learned more about the functional behavior assessment process, including a lecture on Culturally Responsive FBA and Behavior Intervention Planning (BIP) based on an article written by Jimenez-Gomez and Beaulieu in 2022. All students taking the FBA course were exposed to this content in week 7 of the 8-week semester. The addition of course-specific content on cultural and ethnic diversity served as a second exposure to cultural and ethnic diversity content for those students who had previously taken the cultural competency course and an initial exposure for the students who had not yet completed the cultural competency course.

Case Study: Zeb is a 7-year-old boy diagnosed with autism and bi-polar disorder. Zeb currently attends a public school in his neighborhood and is placed in a 12:1:1 classroom which means there are 12 students, 1 special education teacher, and 1 classroom aide in his classroom. Zeb’s behavior has been escalating over the last month and his teachers report that he is getting into fights with other students 2–3 days per week.

It is near the end of the school year and, as such, there are many celebrations and special events. All of the students with summer birthdays are invited to bring in a treat for the class to celebrate in June. Zeb does not like changes to the schedule. He also adheres to a very strict diet and usually does not eat the treats that are brought in for parties or celebrations.

When there is a party or celebration in class, if Zeb sees the treats, he will often become upset and hit or kick the student who has them. In order to avoid the issue, the classroom aide takes Zeb to another room to participate in other, preferred activities.

• As in the first assignment, students were asked to identify general ecological or setting events that could influence the sensitivity of the problem behaviors to the maintaining contingencies and to develop one additional question to obtain more information about relevant ecological or setting events.

• The following ecological or setting events were part of the scenario.

• change of schedule

• medication effects/dual diagnosis

• peer relationships/bullying.

• dietary restrictions (cultural variable)

• For the purposes of course evaluation, student responses were re-analyzed to determine how many students identified and wrote a question about dietary restriction as a setting event. Food and diet are specifically listed as an important part of culture in the cultural competency course. It is also a cultural consideration they learn about in the culturally responsive FBA lecture of the FBA course. For students who have previously taken the cultural competency, we can see the effects of multiple exposures of cultural and ethnic diversity content. For students who have only received cultural and ethnic diversity content as a part of the FBA course, it serves as an AB comparison over the baseline presentation at the beginning of the semester.

Author Contributions

Conceptualization, V.P., M.F.; methodology, V.P., M.F., M.S., and D.N..; formal analysis, V.P., M.F.; data curation, V.P., and D.N.; writing- original draft preparation, V.P., and M.F.; writing- review and editing, V.P., M.F., D.N., and M.S.; visualization, V.P.; project administration, V.P., and D.N.. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Declarations

Author Note

The first and second authors are faculty members in the program where the evaluation occurred. We have no other known conflict of interest to disclose.