Behavior Analysts’ Relationship to Relating Relations: A Survey on Perceptions, Acceptability, Knowledge, and Capacity for Derived Stimulus Relations Research and Practice

Abstract

The study and application of procedures that result in stimulus relations via relational frame theory (RFT) and stimulus equivalence (applied as equivalence-based instruction; EBI), have made tremendous strides in contemporary behavior analysis. However, applications at scale lag basic and translational research. We turn our attention inward to investigate potential causes. We replicated and extended Enoch and Nicholson (Behavior Analysis in Practice, 13(3), 609–617, 2020) by conducting a survey of behavior analysts (n = 129) to determine their perceptions, experiences, and barriers in carrying out research and practice based on RFT and EBI. Participants indicated an interest in RFT and EBI, and mostly perceive both within the scope of behavior analysis. A majority of behavior analysts reported formal education in EBI (78.3%), in contrast to a minority in RFT (15.5%). Adoption of procedures derived from RFT and EBI may be in proportion to formal education. Compounded with a lack of accuracy on basic knowledge questions, there is a potential gap in capacity in the field in addressing behavior related to complex verbal behavior.

Employing efficient and scalable interventions is vital to address issues of great social importance. Such interventions maximize the amount learned, with the least possible time and effort possible (e.g., well-structured online skill building programs that can be completed anytime and anywhere; Critchfield, 2018). Instructional programming that fosters emergent relations may be an efficient instructional technology that fits this niche well, unfortunately, applications that make use of emergent learning are underused in behavior analysis (see Dixon et al., 2018, for detailed arguments in this vein). Two contemporary behavioral theories of human language are stimulus equivalence (applied as equivalence-based instruction [EBI]; Sidman, 1971) and relational frame theory (RFT; Hayes et al., 2001). Both lead to emergent learning and shed light on the symbolic and referential nature of language (see Critchfield et al., 2018, regarding the historical and contemporary accounts of these theories).

Accumulating evidence, across disciplines, appears to converge on the finding that relational responding is key to language and learning (e.g., McLoughlin et al., 2020). A benefit of conceptualizing language instruction using relational responding is a precise account of the teaching arrangements that might yield specific outcomes (Critchfield, 2018). For example, in an EBI procedure used in higher education, instructors may teach foundational skills such as identifying a definition (A) when presented with a term (B) via direct contingencies (resulting in A = B relations). It is expected that students would derive the correct term (B) when presented with the definition (A) without direct training (resulting B = A relations), in the absence of a contingency. If a student is then taught that the term (B) can be depicted in graphical form (C) via direct contingencies (resulting in B = C relations), they are also likely to derive the definition (A) when presented with the graph (C), also without further training (A = C relations; e.g., Lovett et al., 2011; Walker & Rehfeldt, 2012). Additional training based on RFT may result in deriving relations such as the use of the term in a novel situation (i.e., transfer or transformation of function; see Dymond & Rehfeldt, 2000, for a description of these processes), and how this term may be distinct, more or less applicable, or part of a broader theoretical relationship (see Dixon & Stanley, 2020, for an easy-to-read description of relational framing). Instructional procedures that make use of the above methods are, by definition, efficient, because just a few skills are directly taught, and many relations are derived.

EBI and RFT research provides instructional technologies that can be used in applied clinical settings (see Belisle et al., 2020; Critchfield & Rehfeldt, 2020; Pilgrim, 2020) and higher education (see Brodsky & Fienup, 2018; Critchfield et al., 2018). Both methods have been studied across a broad spectrum of subject matter. For example, EBI has been applied in clinical settings to increase verbal behavior (e.g., Groskreutz et al., 2010) and reading comprehension using activity schedules (e.g., Miguel et al., 2009); and RFT-based instruction has been applied in training perspective-taking skills (e.g., Jackson et al., 2014) and metaphorical reasoning (e.g., Persicke et al., 2012) with children with autism. In addition, EBI has been applied in higher education by increasing knowledge of operant functions of behavior (e.g., Albright et al., 2016), neuroanatomy (e.g., Fienup et al., 2016), and logical fallacies (e.g., Ong et al., 2018); and RFT-based instruction has been applied in training trigonometric transformations (e.g., Ninness et al., 2009) and statistical transformations (e.g., Sandoz & Hebert, 2017) with college students. Further, derived stimulus relations research has been named as holding great promise in research on societal issues such as racism (e.g., Matsuda et al., 2020), sustainability (e.g., Biglan & Barnes-Holmes, 2015), and in organizations at large (Hayes et al., 2006).

Underrepresentation of Derived Stimulus Relations Research in Behavior Analysis Journals

In general, there is an impressive cumulative literature base of EBI and RFT research in behavior analytic journals (see Barnes‐Holmes & Harte, 2022; Belisle et al., 2020; Rehfeldt, 2011). Despite great promise, EBI and RFT research is currently relatively underrepresented as a focus of behavior analytic research. To illustrate the lack of allocation of research efforts toward these ends, we conducted a search for publications that contained the author keywords “stimulus equivalence” and “relational frame theory” over the past 5 years via Web of Science (the search was conducted at the time of writing—October 2022). We subsequently sorted these findings by publication title (i.e., journal name) and searched for the total number of articles published per outlet. Our results revealed derived stimulus relations research is most commonly published outside of the two journals that are considered the most prominent and widely read by applied researchers and practitioners in the field—Behavior Analysis in Practice (BAP) and the Journal of Applied Behavior Analysis (JABA). Within the past 5 years, stimulus equivalence appeared as a keyword in 11 articles in JABA, accounting for just 2.18% of the total articles published; RFT appeared as a keyword in five articles in JABA, accounting for just 0.99% of the total articles published. A similar proportion of articles was found in BAP, though there were notably more RFT-related articles; stimulus equivalence appeared as a keyword in 10 articles, which accounts for just 1.81% of the total articles published; RFT appeared as a keyword in 22 articles and accounts for just 3.99% of the total articles published. The most common outlet for both stimulus equivalence and RFT was The Psychological Record; stimulus equivalence appeared as a keyword in 63 articles, accounting for 23.51% of the total articles published; RFT appeared as a keyword in 38 articles, accounting for 14.18% of the total articles published (see Table 1 for the full results of our search). The second most common outlets were the Journal of the Experimental Analysis of Behavior and the Journal of Contextual Behavioral Science for EBI and RFT, respectively. Both of the above journals publish research that is certainly of interest to the applied wing of behavior analysis, however, applied professionals in the field may not frequently attend to the content of these journals. In sum, the percentage of total articles published in the most common publication outlets for these keywords ranged from 0.05% to 14.18% for RFT and 1.18%–23.51% for stimulus equivalence. Neither may be expected to make up most of the research in any publication outlet, nor would this be beneficial; a professional outlet exclusively based one theory or procedure would limit the exposure of behavior analysts to the topic, if the readership is not widespread across journals. Given the above search results, it appears that the influence of both EBI and RFT lines of research pale in comparison to the proportion of other topics published in the behavior science literature, which begs the question—“why?”

Table 1.

Top Five Derived Stimulus Relations Research Publication Outlets

	Published articles with keyword	Total Articles Published	Percent of Total Articles Published
Relational Frame Theory
Psychological Record	38	268	14.18
Journal of Contextual Behavioral Science	31	416	7.45
Behavior Analysis in Practice	22	552	3.99
Perspectives on Behavior Science	13	221	5.88
Frontiers in Psychology	11	21,133	0.05
Journal of Applied Behavior Analysis*	5	505	0.99
Stimulus Equivalence
Psychological Record	63	268	23.51
Journal of the Experimental Analysis of Behavior	46	358	12.85
Perspectives on Behavior Science	17	221	7.69
Journal of Applied Behavior Analysis	11	505	2.18
Behavior Analysis in Practice	10	552	1.81
Behavioral Processes	10	850	1.18

The data above represents publications over the past 5 years. *JABA was included in RFT data as a point of comparison. The journal was ranked as the 8^th most common outlet to include RFT as an author keyword

To answer the above question, a suitable starting point may be an investigation of the perceptions of whether EBI and RFT are within the scope of practice for the field of behavior analysis and whether individual behavior analysts feel competent in these areas. Most behavior analysts surveyed by Enoch and Nicholson (2020) indicated an interest in RFT (i.e., 90.76% were interested in reading peer-reviewed research; note: Enoch and Nicholson did not include EBI in their survey; hence, the status of EBI is unknown in all of the following domains). A minority of respondents (22.44%), however, reported the inclusion of the topic in their formal educational experiences. Given the limited number of formal RFT educational opportunities Enoch and Nicholson reported, the lack of adoption of RFT methodology in respondents’ professional practice is unsurprising. Approximately 40% of participants indicated that they foresaw challenges in implementing behavioral interventions based on RFT; further, most participants indicated either “no” (11.88%) or “low” (50.50%) understanding of RFT as an account of language. Additional open-ended questions identified barriers such as a lack of understanding and application of the theory, and lack of adequate training (Enoch & Nicholson, 2020). Taken together with the limitations of the self-report nature of the study (i.e., perspectives on self-competence do not necessarily reflect competency in practice); individuals that have minimal prerequisite knowledge may not foresee the difficulty in carrying out these methods.

Further, Enoch and Nicholson’s (2020) results must be considered in light of the fact that the survey was conducted prior to the COVID-19 pandemic, in which many practitioners were forced to move into telehealth models of instructional delivery, which may present additional technological challenges in delivering instruction, not mentioned in the survey. Derived stimulus relations research and practice is well-suited for computer-based instruction, which is possible via a variety of formats (e.g., PowerPoint, Adobe Captivate, Google Forms and Slides; Blair & Shawler, 2019). The above formats free instructors from face-to-face, one-to-one instruction, in real time. Meaning that more individuals may benefit from effective instruction in less time. Unfortunately, the above noted modes of instructional delivery pose barriers such as cost, access to technology, and technical skills (e.g., coding). Previous studies, such as Blair and Shawler (2019), brought attention to the unfortunate state of EBI and RFT computer-based research; in a review of 39 research studies that made use of technology to deliver derived stimulus relations tasks, none of the studies met the Baer et al. (1968) “technological” description criteria. Blair and Shawler (2019) did attempt to remediate this issue by providing step-by-step instructions using low-cost alternatives, powered by Google. However, these platforms pose different limitations, such as a lack of automated data collection and immediate feedback, and implementation data on this set of instructions does not exist, to date. In the end, the findings of the current computer-based derived relations research, though meant to be applied research, are at best translational, and have utility at the conceptual rather than practical level. Compounded with a lack of formal education, the above issues may result in a context in which theories with great potential to “save the world” underperform.

A proposed model for contemporary behavior science research is a reticulated model, in which each level (basic, translational, applied, and practice) must inform and influence others in a reciprocal fashion (Villatte et al., 2018; Hayes et al., 2021). Hayes et al. (2021) state that “the best basic science allows us to simultaneously understand, predict, and influence change in the real world, and the best applied program readily links to and aids in a fuller understanding and specification of basic principles” (p. 181). This strategy can be interpreted as a “30,000-foot view” with the aim of advancing science, in which progress is evaluated based on the practical outcomes it achieves (i.e., influencing behavior to improve people’s lives). Meaning that barriers to applications of basic research findings must be minimized in a reticulated model. Applied researchers and practitioners must be able to readily make use of the literature base to further expand it. To make the above a reality, Dixon et al. (2018) suggested that RFT researchers must turn their attention inward to examine the likely cause for a lack of adoption of their methods at scales likely to influence problems of great social importance. Thus, the purpose of this study is to identify barriers to conducting derived stimulus relations research and practice.

Method

Participants and Materials

Participants were invited to participate in an anonymous survey via the Qualtrics survey platform. Participants were recruited using the Behavior Analyst Certification Board (BACB) mass email service. Recruitment took place during May and June 2021. Participation was limited to master’s and doctorate-level board certified behavior analysts (BCBA and BCBA-D certificants, respectively), located in either Canada or the United States, who were over 18 years old. Recruitment emails were sent to 19,711 subscribers of the BACB mass email service in the United States and Canada, who opted in to receive solicitation emails from the BACB; 5,959 (30.26%) unique subscribers opened the email, and 13,736 (69.68%) subscribers did not; 16 emails could not be delivered; and three people unsubscribed from the service upon receipt of this email (see information regarding the BACB Mass Email Service here: https://www.bacb.com/mass-email-information). A total of 353 (5.92%) subscribers clicked a link to be directed to a screening questionnaire (i.e., participants were asked three questions to determine if they met the inclusion criteria).

The screening questionnaire asked whether respondents were (1) over the age of 18; (2) whether they reside in Canada or the United States; and (3) whether they were certified at either the master’s or doctorate levels as board certified behavior analysts. Those that indicated that they did not meet all of the above criteria were not included in the analysis and were directed to a page that thanked them for their interest in the study. If the above criteria were met, a letter of information and consent was presented.

Survey respondents that initially provided consent included 196 individuals; however, 67 participants did not complete the survey. Responses were not included in the analysis if the survey was exited prior to completion. Participants were free to skip questions. Questionnaires were defined as complete if participants were presented with all questions in the survey. Of the 67 participants that did not complete the survey, 63 exited the survey when they encountered the knowledge question portion. Only four participants exited the survey at another point. The sample included in the analysis contained 129 participants (65.81% of the initial sample who consented to participate completed the survey).

Upon completing the survey, participants were given the option to be included in a random raffle drawing for a $100 gift card.

Instrumentation

Part I of the survey presented 12 demographic questions (see demographic results in Table 2). Participants were asked to indicate their age, gender, geographic region in which they reside, professional experience and credentials, education, and area of specialization. Participants were asked questions regarding their familiarity with and the acceptability of derived stimulus relations (one open-ended, 17 yes/no, 13 five-point Likert scales questions). This section replicated and extended the survey conducted by Enoch and Nicholson (2020), with the addition of questions related to stimulus equivalence. The section also asked questions specific to carrying out RFT and or equivalence research and practice using coding (e.g., E-Prime, Python, PsychoPy, JavaScript) and noncoding platforms (e.g., learning management systems such as Blackboard, Desire2Learn).

Table 2.

Reported Demographic Characteristics

	n	%
Gender
Female	109	50
Male	18	13.9
Prefer not to answer	2	1.6
Age
22–25	5	3.9
26–30	31	24.0
31–35	28	21.7
36–40	31	24.0
41–45	11	8.5
46–50	8	6.2
51–54	7	5.4
55–59	1	0.8
60–64	4	3.1
65 +	3	2.3
Country
United States	106	82.2
Canada	23	17.8
Certification
Master's degree	103	79.8
Doctoral degree	26	20.2
Years Certified
0–5	72	55.8
6–10	40	31.0
11–15	12	9.3
15–20	4	3.1
20+	1	0.8
BACB Task List
1	1	0.8
2	9	7.0
3	29	22.7
4	89	69.5
Degree Specialization
Education	24	18.6
Behavior Analysis	88	68.2
Psychology	13	10.1
Other	4	3.1
Professional role
Academia	15	11.6
Administrator	8	6.2
Applied researcher	3	2.3
Practitioner / clinician	95	73.6
Student	4	3.1
Other	4	3.1
Practice setting
Center/clinic	34	26.4
Clients' homes	34	26.4
College/university	19	14.7
Community	2	1.6
Hospital	4	3.1
Private school	3	2.3
Public school	19	14.7
Residential facility	7	5.4
Other	7	5.4
Population served
Autism spectrum disorder	81	62.8
Emotional or behavioural disorders	9	7.0
General education	2	1.6
Gerontology	1	0.8
Intellectual disability	7	5.4
Mental health	1	0.8
Other	4	3.1
Special education	10	7.8
Students (higher education)	14	10.9

Next, participants completed a six-item matching task and three multiple-choice questions to gauge participants’ knowledge related to derived stimulus relations. Questions were asked regarding terms and concepts (e.g., identifying the definition when presented a term). Questions were drawn from the instructor’s manual (Allison et al., 2020) that accompanies the Applied Behavior Analysis (3rd ed.) textbook by Cooper et al. (2020). Finally, participants indicated resources that might be helpful in implementing derived stimulus relations programming, in an open-ended question format.

Results

Respondent Demographics

One hundred nine participants identified as female, 18 identified as male, and 2 preferred not to answer. Age ranged from 22 to above 65 (M = 37.3; SD = 9.83). Certification status of participants included 20 BCBA-D and 109 BCBA certificants. Participants identified their location by country; participants were located in either the United States (n = 106) or Canada (n = 23). Additional participant characteristics were indicated, including the BACB task list that served as the basis for their certification examination (first: n = 1; second: n = 9, third: n = 29, and fourth: n = 90) and the number of years since receiving their highest degree, which ranged from 1 – 32 (M= 7, SD = 5.26). In cases that participants indicated that they did not know which BACB task list was in use at the time of their certification (n = 27), the task list was inferred based on the number of years since receiving their highest degree and years certified as a BCBA (e.g., based on the dates each task list was in use relative to the likely year a participant wrote the certification exam). Participants also indicated their primary population served; 62.8% of participants served individuals with autism spectrum disorder, other populations served included students in higher education (10.9%), special education (7.8%), and emotional or behavioral disorders (7.0%; see Table 2 for the distribution across practice settings and populations served).

RFT and EBI Experience and Perceptions

A total of 67.4% participants indicated that they did not complete coursework in RFT (see Table 3 for all experience and perceptions results), conversely 78.3% of participants indicated that they did complete required coursework in EBI. Likewise, 84.5% of participants indicated a lack of availability of elective coursework in RFT; 74.4% of participants also reported a lack of elective coursework in EBI. Participants indicated an interest in reading both RFT and EBI research (RFT: 70.6%; EBI: 86.6%), continuing education opportunities (RFT: 89.1%; EBI: 89.1%) and reading research outside of course requirements (RFT: 69.8%; EBI: 72.1%). Fewer participants reported participating in continuing education (RFT: 53.5%; EBI: 47.3%). When asked about incorporating RFT and EBI into their current research and practice, 37.2% of participants reported incorporating RFT; whereas 72.1% reported incorporating EBI. Trends were similar for reports of whether participants’ colleagues incorporated RFT and EBI into their research and practice, though fewer colleagues were reported to do so (RFT: 33.3%; EBI: 54.7%). When asked to rate their understanding of RFT and EBI as a behavior-analytic account of complex verbal behavior, participants indicated lesser understanding of RFT than EBI, but indicated a moderate to very high understanding of both (RFT: 60.4%; EBI: 86%). Likewise, participants reported greater acceptability of EBI relative to RFT as a theory within the scope of applied behavior analysis (RFT: 80.7%; EBI: 96.2%), and as a theory within their personal scope of competence (RFT: 60.5%; EBI: 83.7%). When asked about the potential of RFT and EBI to inform their research and practice, a similar pattern was reported; more participants indicated a likelihood of “likely or extremely likely” for EBI relative to RFT (RFT: 58.2%; EBI: 78.3%). More participants reported they were “likely” or “extremely likely” to face challenges with implementing RFT relative to EBI in their setting (RFT: 41.1%; EBI: 25.6%). A total of 26.4% of participants reported that they have access to a learning management system (LMS). Only 6.3% of participants reported “moderate,” “high,” or “very high” personal competence with using a software coding language (e.g., JAVA, Visual Basic, E-Prime) to program a derived stimulus relations task.

Table 3.

Familiarity, Perceptions, and Acceptability of Derived Stimulus Relations

Survey Question	Counts	% of Total
Required coursework in RFT
No	87	67.4
Yes	42	32.6
Required coursework in EBI
No	28	21.7
Yes	101	78.3
Elective coursework in RFT
No	109	84.5
Yes	20	15.5
Elective coursework in EBI
No	96	74.4
Yes	33	25.6
Interested in reading RFT research
Strongly disagree	6	4.7
Disagree	6	4.7
Neutral	26	20.2
Agree	53	41.1
Strongly agree	38	29.5
Interested in reading EBI research
Strongly disagree	3	2.3
Disagree	4	3.1
Neutral	18	14.0
Agree	68	52.7
Strongly agree	36	27.9
Interested in RFT continuing education
No	14	10.9
Yes	115	89.1
Interested in EBI continuing education
No	14	10.9
Yes	115	89.1
Read research on RFT (outside of education)
No	39	30.2
Yes	90	69.8
Read research on EBI (outside of education)
No	36	27.9
Yes	93	72.1
Participated in RFT continuing education
No	60	46.5
Yes	69	53.5
Participated in EBI continuing education
No	68	52.7
Yes	61	47.3
Currently incorporate RFT into practice/research
No	81	62.8
Yes	48	37.2
Currently incorporate EBI into practice/research
No	36	27.9
Yes	93	72.1
Colleagues incorporate RFT into practice/research
No	86	66.7
Yes	43	33.3
Colleagues incorporate EBI into practice/research
No	58	45.3
Yes	70	54.7
Understanding of RFT
none	9	7.0
low	42	32.6
moderate	59	45.7
high	15	11.6
very high	4	3.1
Understanding of EBI
none	1	0.8
low	17	13.2
moderate	67	51.9
high	36	27.9
very high	8	6.2
Acceptability of RFT (scope ABA)
none	6	4.7
low	19	14.7
moderate	40	31.0
high	42	32.6
very high	22	17.1
Acceptability of EBI (scope ABA)
none	1	0.8
low	4	3.1
moderate	22	17.1
high	62	48.1
very high	40	31.0
Acceptability of RFT (personal scope of competence)
none	11	8.5
low	40	31.0
moderate	41	31.8
high	22	17.1
very high	15	11.6
Acceptability of EBI (personal scope of competence)
none	4	3.1
low	17	13.2
moderate	45	34.9
high	43	33.3
very high	20	15.5
Potential of RFT to inform your research & practice
Extremely unlikely	10	7.8
Unlikely	12	9.3
Neutral	32	24.8
Likely	49	38.0
Extremely likely	26	20.2
Potential of EBI to inform your research & practice
Extremely unlikely	2	1.6
Unlikely	9	7.0
Neutral	17	13.2
Likely	71	55.0
Extremely likely	30	23.3
Likely to face challenges implementing RFT in setting
Extremely likely	10	7.8
Likely	43	33.3
Neutral	42	32.6
Unlikely	25	19.4
Extremely unlikely	9	7.0
Likely to face challenges implementing EBI in setting
Extremely likely	5	3.9
Likely	28	21.7
Neutral	38	29.5
Unlikely	43	33.3
Extremely unlikely	15	11.6
Access to LMS
No	95	73.6
Yes	34	26.4
Personal competence w/ coding to program DRR Task
None	91	70.5
Low	30	23.3
Moderate	6	4.7
High	1	0.8
Very high	1	0.8

Finally, we analyzed the proportion of participants that reported formal education in EBI and RFT and whether these participants reported incorporating these procedures into their practice (see Fig. 1). Of the 101 participants who had required EBI coursework, 77 (76.24%) incorporated EBI into their practice and 24 (23.76%) did not. Of the 28 who did not have required EBI coursework, 16 (57.14%) incorporated EBI into their practice and 12 (42.86%) did not. Of the 42 participants who had required RFT coursework, 21 (50%) incorporated RFT into their practice, and 21 (50%) did not. Of the 87 who did not have required RFT coursework, 27 (31.10%) incorporated RFT into their practice and 60 (68.97%) did not.

Fig. 1.

Required Coursework and Incorporating Procedures into Practice. Note. The left panel represents the number of participants that reported required coursework in EBI, the right panel represents the number of participants that reported required coursework in RFT. Both represent the number of participants who incorporate these procedures into their practice

RFT and EBI Knowledge

A majority of participants lacked fundamental knowledge of both EBI and RFT terms, concepts, and examples as assessed by the matching and multiple-choice questions taken from the Allison et al. (2020) instruction manual. Questions were scored as correct or incorrect according to the answer key supplied in the instruction manual.1 Participants certified at the doctoral level responded more accurately on both matching questions (BCBA: n = 100, M = 37.3%; BCBA-D: n = 19, M = 48.2%; see Table 4 and Fig. 2) and multiple-choice questions (BCBA: n = 108, M = 33.8%; BCBA-D: n = 20, M = 46.7%).

Table 4.

Knowledge Question Accuracy across Certification Levels

	Certifications	N	Missing	Mean	Median	Minimum	Maximum
Matching %	BCBA	100	9	37.3	33.3	0.00	100.0
	BCBA-D	19	1	48.2	50.0	0.00	100.0
Multiple Choice %	BCBA	108	1	33.8	33.3	0.00	83.3
	BCBA-D	20	0	46.7	50.0	0.00	100.0

Fig. 2.

Knowledge Multiple Questions and Matching Question Accuracy across Certificant Level. Note. The left panel represents multiple-choice question accuracy, the right panel represents matching question accuracy. Each data point represents a participant’s score. The black square represents the mean. The width of each curve represents the approximate frequency of data points in each region

We also analyzed the accuracy of responses by participants who reported incorporating EBI and RFT into their research and practice, related to each subject matter. Participants who completed all EBI-related knowledge questions, who incorporate EBI (n = 87), had a mean accuracy of 37.5%. Participants who completed all RFT-related knowledge questions, who also incorporate RFT (n = 45), had a mean accuracy of 44.4% (see Fig. 3).

Fig. 3.

Knowledge Question Scores Related to EBI and RFT Subject Matter and Incorporating into Research and Practice. Note. The top panel represents knowledge questions related to EBI and the bottom panel represents knowledge questions related to RFT. Each data point represents a participant’s percentage of correct responses. The black square represents the mean. The width of each curve represents the approximate frequency of data points in each region

Open-Ended Question Responses

Ninety-six participants responded to the open-ended question to indicate support needed in carrying out derived stimulus relations programming (e.g., training in coding, task analyses, annotated protocols). 46 participants indicated a need for training in coding and other technological barriers; 27 participants indicated that they require additional training to acquire a conceptual understanding; and 3 participants indicated workplace issues (e.g., a lack of autonomy to make decisions related to programming for their clients).

Discussion

The present study sought to determine the barriers behavior analysts may face in carrying out derived stimulus relations research and practice. The study replicated Enoch and Nicholson’s (2020) study on the acceptability and perspectives on acceptance and commitment therapy and RFT. We also extended their findings by including EBI, assessing knowledge of EBI and RFT, and focusing on technology. We found that participants reported perceptions of and experience with RFT that were mostly consistent with Enoch and Nicholson’s (2020) findings (i.e., a mostly favorable view of RFT and EBI). The addition of questions related to EBI shed light on discrepancies within educational experiences and ongoing research and practice in the field (i.e., EBI is taught and practiced far more than RFT). We found that perceptions of competence were incongruent with a direct assessment of basic knowledge of both EBI and RFT in our sample (i.e., confidence did not yield high accuracy). This can be illustrated by the fact that most participants indicated a moderate to high understanding of both EBI and RFT, and similarly endorsed both as being within their personal scope of competence, yet the minority accurately responded to basic knowledge questions. The above pattern also held for participants that reported incorporating EBI and RFT into their research and practice. The fact that participants that apply procedures based on EBI and RFT, but do not appear to have some of the requisite knowledge to do so is a major point of concern due to the potential for risk of harm. Finally, some questions were structured to allow inference regarding competence in carrying out computer-based instruction (i.e., participants lacked computer programming skills and access to a learning management system; and require training and resources).

A reasonable assumption is that competence in a complex topic must start with education. We found that most behavior analysts surveyed are interested in RFT (i.e., reading journal articles and attending continuing education events) but lack formal education; this finding is consistent with Enoch and Nicholson (2020). A minority of participants reported that they incorporate RFT into their research and practice, conversely a majority of those with formal education in EBI reported that they incorporate EBI into their research and practice. Faculty-level discretion regarding BACB Task List items such as 4^th Edition Task List item J-14: “Arrange instructional procedures to promote generative learning (i.e., derived relations)” and 5^th Edition Task List item B-15: “Define and provide examples of derived stimulus relations” may be a root cause of the discrepancy in our results. Faculty may cover EBI, RFT, or both. However, 4^th Edition Task List item E-06: “Use stimulus equivalence procedures” and 5^th Edition Task List item G-12: “Use equivalence-based instruction” may tip the scales in favor of EBI. Further, future faculty are products this exact system of education, and are likely to continue the cycle until contingencies are shifted toward the inclusion of RFT. The 6^th Edition Task List only contains a singular item - B-21: “Identify examples of processes that promote emergent relations and generative performance”; we are hopeful that instructors will decide to incorporate RFT into their lessons; otherwise, future practitioners with an interest in RFT will continue to need to seek out their own education, to be equipped to analyze behavior involving relations beyond equivalence. Unfortunately, barriers such as cost, and the investment of time required to grasp a novel area of study, especially upon graduation from higher education, may result in a lack of adoption of this methodology, and a limited personal scope of practice in solving problems of greater complexity.

We found inconsistent results with Enoch and Nicholson (2020) in perceptions of personal scope of competence; most participants indicated that RFT is within their scope rather than a minority. A higher proportion of participants endorsed this view for EBI. Though these perceptions need to be interpreted with caution, because most participants were unable to correctly respond to knowledge questions related to EBI and RFT, based on the gold standard textbook in the field of behavior analysis (i.e., Cooper et al., 2020). The average percentage of correct responses was approximately 50% or less. Hence, the relationship between more formal education, perception, basic knowledge, and competence in practice requires further investigation. In addition, experience conducting research and practice (i.e., practical on-the-job practical experience) in these areas did not seem to influence accurate responding to knowledge questions; this is evident in the similarly poor accuracy across participants that do and do not make use of EBI and RFT methods. It is possible that multiple-choice and matching questions may not be predictive of competence as a practitioner or researcher. Though this type of assessment holds some face validity—it may be difficult to plan an intervention without foundational conceptual knowledge. For example, to effectively plan instruction using EBI and RFT procedures, the ability to accurately identify the types of relational responses involved among the stimuli taught is a critical skill. Unfortunately, the above skill appeared to be missing in a substantial portion of our sample, as evaluated by our knowledge question results. Hence, additional education is required to provide behavior analysts with the best start in the field possible. In addition, and most important, practitioners may be at risk of applying procedures outside of their personal scope of competence, thereby, putting the people they serve at risk of harm via a lack of effective practice, and thereby potentially damaging the reputation of the field.

Likewise, a gap in the preparation of behavior analysts to scale RFT and EBI methodologies is implied by survey questions that illustrate competence in programming computer-based tasks. In-person, paper–pencil, one-to-one instruction inherently limits the number of individuals that may benefit from it. The workforce in behavior analysis is simply not big enough to teach socially valid skills to all the people that may benefit from them. Online instruction in education and clinical practice using derived stimulus relations methodology is inherently efficient (Critchfield, 2018), but will continue to lack external validity as a methodology unless it is widely applied. The fact that EBI and RFT literature lacks technological descriptions of preparing computer-based tasks may be compounded with a lack of access to learning management systems (Blair & Shawler, 2019), which is especially problematic given that a slim portion of behavior analysts report personal competence with using a coding language (e.g., 6.3% of the sample in the present study).

Given the above, the field likely lacks capacity to meaningfully scale RFT and EBI interventions beyond the point of basic, translational, and small-scale implementation studies. Some behavior analysts are actively working to build the capacity of behavior analysts to code in general (i.e., codingbehavioranalysts.org), which is encouraging, but is mostly done on a volunteer basis, outside of formal education. Scaling behavior analytic interventions likely requires serious consideration of incorporating these skills into behavior analytic methods of instruction.

In short, given the assumption that our sample is representative of the field, we believe we have identified substantial gaps in the educational and potentially practical preparation of behavior analysts. Further, the participants in Enoch and Nicholson (2020) may have naïvely indicated a lack of barriers to carrying out RFT-based instruction, because similar gaps in knowledge to those found in our sample likely exist in that sample as well. In other words, the survey respondents may not know what they do not know because those that lack minimal prerequisite knowledge may not foresee the difficulty in programming tasks from both a theoretical and/or a technological perspective. Hence, practitioners and applied researchers may benefit from additional training, resources, and materials that address practical knowledge gaps. One solution may lie in increased efforts to publish special issues in journals that are routinely read by applied professionals (i.e., JABA and BAP) on topics that not only span the applications of RFT and EBI, along with specific populations that may benefit, but also specific applications with an eye toward scale (e.g., online and computer-based instruction, and artificial intelligence and machine learning methods). The above may be a first step in addressing some of relative lack of allocation of EBI and RFT in these outlets described above.

Having said all the above, our study was not without limitation. Selection bias may have influenced the results. Only 5.92% (n = 353) received the email and clicked through to access the screening questionnaire, whereas 94.08% (n = 5,606) opened the email and did not take any action. Further 13,736 emails remained unopened out of a total of 19,711 unique subscribers across the BACB mass email distribution list. Though the open and click rates appear relatively low, these metrics are higher than the average email open and click rates across industries analyzed by Mail Chimp; 21.33% and 2.62%, respectively (Mail Chimp, 2021). Notwithstanding the click-through rate, it is not possible to determine whether participants were highly motivated to participate to either support or diminish the topics of this study. Meaning that further replication is needed to make more confident generalizations to the overall population of behavior analysts. Wider recruitment strategies, such as recruitment via local behavior analysis professional associations may aide in representative recruitment from across regions. The results of the present study did demonstrate reliability across studies by replicating many of the findings of Enoch and Nicholson (2020). However, both studies may be limited by similar limitations related to selection bias and self-report.

An additional limitation is that our study had a relatively high attrition rate (34.18%). It is noteworthy that the majority of participants that dropped out of the study did so upon encountering the knowledge question phase of the study. A plausible explanation for this attrition is the increased response effort and or the perception of the need to accurately answer knowledge questions. Unfortunately, we did not conduct an exit survey, therefore, conclusive information on why participants chose to exit the study is not available, to confirm the above; meaning that other factors cannot be ruled out. If the above hypothesis regarding attrition is accurate, this implies that knowledge question accuracy in this study may be an overestimate of the state of knowledge of RFT and EBI, as represented by the sample in the present study.

Conclusion

Derived stimulus relations play a central role in complex human behavior, and evidence suggests that influencing how people relate stimuli has “a significant impact on human action” (see Dixon & Hayes, 2022, for a recent discussion). If a limited number of behavior analysts are truly prepared to carry out procedures based on EBI and RFT, the personal scope of competence of these professionals will be limited; unfortunately, if most behavior analysts have a limited scope, the scope of the field will be limited in turn, therefore, the number and complexity of problems the field may adequately address will be limited as well. We hope that an awareness of the above issue may be a steppingstone to behavior change in the field. The recent introduction of commercially available packages for training derived relational responding (e.g., PEAK, Dixon, 2015, 2016) are certainly useful additions to the pantheon of methods available to practicing behavior analysts, but the adoption and effective use of those technologies depends upon behavior analysts understanding the science and basic behavioral processes upon which those tools are based. In addition, future research is required at the level of impact to gauge progress on how widely EBI and RFT methods are applied (Dixon et al., 2018). This research will complement research on the effectiveness of EBI and RFT and demonstrate the broad applicability of these procedures in addressing socially important issues.

Trainers and educators hold the extraordinarily important role of facilitating learning that influences future practice and professional development. Perhaps more important, instructors are responsible for the outcomes of the people served by practitioners in the field. Therefore, training behavior analysts in theories and procedures that address complex human behavior is essential. It does not appear to be an understatement to claim that the betterment of society rests with instructors charged with the task of establishing pivotal skills (Critchfield, 2018). We encourage instructors and all those that influence the selection of curricula content areas of study in the field to include EBI and RFT in their instructional decisions. We state this in light of the findings that those who were required to take classes on EBI and RFT incorporated those concepts into their research and practice at a higher rate; curricular requirements appear to facilitate incorporating these concepts into the range of effective treatments available for their clients.

We also encourage providing continuing education and publication opportunities to meet the demand for the training needs identified in the current analysis, given the demonstrated gaps in EBI and RFT knowledge. Further, we suggest making use of these methods in the training of behavior analysts to demonstrate the real-world value to students and practitioners (see Brodsky & Fienup, 2018, for examples). EBI and RFT offer much potential for promoting socially significant improvements in people’s lives. We must, however, take the necessary steps to prepare practicing behavior analysts to understand and implement interventions based upon the basic behavioral processes that underpin EBI and RFT-based procedures, if we are to capitalize on that potential.

Data Availability

Data will be made available by the first author on reasonable request.

Declarations

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Conflict of Interest

All authors declare that he/she have no conflict of interest.