Abstract
Research has found equivalence-based instruction (EBI) to be effective and efficient, with recent research extending these finding to individuals diagnosed with autism spectrum disorder (ASD). EBI has also been compared with more traditional approaches to teaching, such as traditional lectures, reading assignments, and video lectures. However, the authors are unaware of any comparisons of EBI to other similar behavior analytic approaches such as discrete trial teaching (DTT). The purpose of the first experiment was to compare EBI to progressive DTT using an adapted alternating treatments design with typically developing adults. Experiment 2 compared the two teaching methods with children diagnosed with ASD. The teaching approaches were evaluated with respect to mastery of trained relations, emergence of untrained relations for the EBI condition, and participant preferences. Results of the two experiments found that both teaching methods were effective, progressive DTT was found to be more efficient for six participants, EBI was found to be more efficient for two participants, and overall progressive DTT was found to be more preferred by participants.
Supplementary Information
The online version contains supplementary material available at 10.1007/s40617-022-00687-8.
Keywords: Stimulus equivalence, Equivalence-based instruction, Discrete trial teaching, Autism spectrum disorder
Research has demonstrated that using conditional discrimination procedures to directly teach certain relations (e.g., AB, BC relations) often leads to the emergence of untrained relations (e.g., AC, CA, CB, BA relations; Cooper et al., 2020; Sidman, 1971). For example, teaching an individual to match the word “wine” to a picture of a bottle of wine and that same picture of a bottle of wine to the word “vino” (i.e., the Spanish word for wine). That same individual will match the word “wine” to the word “vino” without any additional, or direct, teaching. This effect, called stimulus equivalence, has been noted to teach generatively because novel/untrained relations emerge without direct teaching, which can decrease time to skill acquisition (Stanley et al., 2018).
Instructional approaches designed and based on the principles of stimulus equivalence have been referred to as equivalence-based instruction (EBI; Fienup et al., 2010). EBI has been suggested as an efficient, or economical, method of intervention due to the number of skills that can be acquired relative to the number of skills directly taught (Rehfeldt, 2011). That is, in comparison to other instructional approaches, EBI theoretically requires less direct teaching to obtain more skills. However, in a recent review of the literature on EBI in higher education, Brodsky and Fienup (2018) concluded that EBI is an effective procedure but were unable to draw a firm conclusion regarding the efficacy of EBI when compared to other alternative instructional strategies.
Brodsky and Fienup’s (2018) review identified two studies that compared EBI to other instructional approaches. Fienup and Critchfield (2011) compared EBI with complete instruction (i.e., teaching all relations directly) and O’Neill et al. (2015) compared EBI with reading a textbook for teaching academic targets. When reviewing these comparative studies, Brodsky and Fienup found a small effect size for Fienup and Critchfield and a medium effect size for O’Neill et al. Additionally, Fienup and Critchfield (2011) found that students had similar levels of mastery for EBI compared with the condition that taught all relations directly, although EBI was deemed to be more efficient in terms of training time and trials.
Since Brodsky and Fienup’s (2018) review, only one other comparison study has occurred. Ferman et al. (2020) compared EBI with video lectures to increase religious literacy for 10 middle-school students. EBI consisted of a one-to-many training structure to train five 6-member equivalence classes. The video lecture involved the opportunity to complete a worksheet while watching a video lecture about the same stimuli used in the EBI condition. The results demonstrated that all five participants who received EBI formed the equivalence classes following training, while only one of the five participants in the video lecture condition formed the equivalence classes.
EBI has also been evaluated to teach children with autism spectrum disorder (ASD) generative relations. In a review of the research on EBI as it pertains to individuals diagnosed with ASD, McLay et al. (2013) found that of the nine studies reviewed, five reported positive findings and four reported variable findings on the emergence of untrained relations. Due to the variability in the findings, McLay and colleagues concluded that more research is required in order to determine the conditions under which learners with a diagnosis of ASD are more likely to demonstrate the acquisition of untrained relations. In more recent research on EBI with individuals diagnosed with ASD, Dixon et al. (2016) demonstrated the emergence of AC relations when targeting geometry skills with the PEAK-E curriculum. Dixon et al. (2017) replicated and extended those findings and found the PEAK-E curriculum to be effective in training geography skills to individuals diagnosed with ASD. Stanley et al. (2018) used EBI to teach a variety of academic skills to individuals diagnosed with ASD and found the procedures to be effective.
One instructional approach that EBI has not been compared with is discrete trial teaching (DTT; Ghezzi, 2007). DTT is composed of three main components: (a) discriminative stimulus, (b) response from the learner, (c) a consequence provided by the instructor contingent on the learner’s response (Ghezzi, 2007; Leaf et al., 2016). In this sense, DTT has a similar trial makeup when compared with EBI. That is, within in both approaches a stimulus is presented, the participant is given an opportunity to respond, and this is followed by an instructor-delivered consequence. While this considerable overlap exists, there have also been several advances in DTT methodology since its inception. Leaf et al. (2016) discussed many of these advances when outlining guidelines for what the authors termed progressive DTT. These advances have included (a) selecting trial targets and placement of stimuli based on the learner, (b) using natural language during instructions, (c) varying instructions, (d) using flexible prompt fading (Soluaga et al., 2008), (e) using instructive feedback to teach additional skills, and (f) using error correction when needed. Similar to the research on EBI, there have been several studies that have evaluated various components of a progressive approach to DTT in isolation or in combination (e.g., Leaf et al., 2018).
Given the considerable overlap between DTT and EBI, the documented effectiveness of both, and the recent advances in DTT, it seems fruitful to evaluate the relative effectiveness of DTT and EBI. Therefore, the purpose of Experiment 1 was to compare DTT with EBI when teaching several conditional discriminations (i.e., matching female scientists’ names, pictures, and a fact about the female scientist) to four neurotypical adults. The purpose of Experiment 2 was to compare DTT to EBI when teaching conditional discriminations (i.e., state name, state flag, state flower) to four children diagnosed with ASD. In the EBI condition, a linear-series training structure was used to train only AB and BC relations, and in the progressive DTT condition, the interventionist followed Leaf et al.’ (2016) guidelines on progressive DTT to teach the relations. The two instructional approaches were evaluated with respect to class-consistent responding, efficiency (i.e., sessions and teaching time to mastery), and participant preference.
Experiment 1: Neurotypical Adults
Methods
Participants
The participants in Experiment 1 included four neurotypical adults. Table 1 displays the demographic information for the participants. All participants were employees at the clinic in which the study took place, naïve to the purpose of the study, and volunteered to participate in the study. Signed informed consent was obtained from all participants.
Table 1.
Adult Participant Demographic Information
| Name | Age | Sex | Race/Ethnicity | Diagnosis | Education |
|---|---|---|---|---|---|
| Ann | 32 years | Female | White | None | Bachelor’s degree |
| Donna | 28 years | Female | White | None | Master’s degree |
| April | 31 years | Female | White | None | Bachelor’s degree |
| Leslie | 35 years | Female | Chinese-American | None | Master’s degree |
Interventionist
The first author served as the interventionist for all probe and teaching sessions. At the time of the study, the interventionist had an undergraduate degree in applied behavior analysis and a master’s degree in behavior analysis. She had more than 8 years of clinical and research experience in the field of applied behavior analysis working with individuals diagnosed with ASD. She also had clinical and research experience implementing progressive DTT.
Setting
All sessions took place in various rooms located in a private clinic that provides behavioral intervention for individuals diagnosed with ASD. Every room contained a child-sized table, child-sized chairs, educational materials, and adult furniture (e.g., desks, computer chairs).
Targets and Materials
Two sets of three 3-member stimulus classes were taught to each participant. One set was taught using EBI, and the other set was taught using progressive DTT. Given the common underrepresentation of women in behavior analysis (Wesolowski, 2002) and science more broadly (Simon et al., 2017), we selected female pioneers in science and a sample of their contributions as targets. The stimulus classes taught consisted of female scientists’ names, pictures, and a fact about the female scientist (see Table 2).
Table 2.
Adult Participant Targets
During pretest and posttest probes, daily probes (described later), and for the stimulus classes taught with EBI, a binder easel was used in which the sample stimulus and comparison arrays were printed on 8.5-in. × 11-in. sheets of white paper and placed into sheet protectors. All stimuli were printed to appear in a 3-in. × 3-in. box. A black square was taped over the sample stimulus to require the participants engage in an observing response on each trial (i.e., lifting up the black square to reveal the sample stimulus). The pages containing the sample stimulus had a white background with the sample stimulus printed in the middle of the page. On each page containing the comparison stimuli, the array was arranged in two rows, with two or three stimuli in each row for probe sessions, and one or two stimuli in each row for training sessions. In the progressive DTT condition, laminated index cards of the stimuli were used each of which were 3-in. × 3-in. and the experimenter arranged the cards on the table in front of the participant at her discretion (e.g., array could include other stimulus class members, arranged in vertical pairs, single stimulus in array).
Training sets were counterbalanced across participants. For Ann and Donna, set one and two were assigned to the EBI and progressive DTT condition, respectively. For April and Leslie, set one and two were assigned to the progressive DTT and EBI condition, respectively.
Dependent Variables
Dependent variables included the percentage of correct responding on pretest/posttest probes, daily probes, and maintenance probes. Correct responding was defined as the participant touching the comparison stimulus that corresponded with the sample stimulus for that relation within 15 s in the absence of any prompting or programmed reinforcement. Pretest/posttest probes were used to measure and analyze the emergent relations. Daily probes were used to assess the trained relations (i.e., AB & BC) in the EBI condition.
Ancillary Measures
Efficiency
Efficiency of the teaching procedures was assessed via the number of teaching sessions and teaching time to mastery. Teaching sessions to mastery was calculated by counting the number of teaching sessions until the mastery criterion was reached (i.e., 100% correct responses across three consecutive daily probes). Teaching time to mastery was calculated by summing the total teaching time for all sessions per teaching condition. For the EBI condition, teaching time for each session began once the participant engaged in the first observing response and ended once the interventionist provided the programmed consequence for the last trial of the session. For the progressive DTT condition, teaching time for each session began once the interventionist provided the instruction for the first trial and ended once the interventionist provided the programmed consequence for the last trial of the session.
Social Validity
To assess social validity a questionnaire with nine questions was provided to participants at the end of the study. Seven questions used a Likert-scale from 1 (i.e., Don’t prefer at all, Not at all appropriate, Not at all effective, and Not at all important) to 5 (i.e., Strongly prefer, Very appropriate, Very effective, and Very important). Two questions were choices between the two conditions. The conditions were given random names (i.e., zot and wob) that were used to label the conditions during teaching sessions. A laminated card with the condition name was placed on the table during each teaching session and was also vocally stated by the interventionist during each teaching session (e.g., “Now we are going to do the wob condition.”). For the two questions in the questionnaire that were choices between the two teaching conditions the random name was used for the adult participant to choose their preference. Below each question in the questionnaire there was space for participants to write additional comments.
Progressive DTT Teaching Session Analysis
Prior researchers have noted challenges for researchers when using flexible approaches to teaching such as a progressive approach to DTT (i.e., Cihon et al., 2020; Cihon et al., 2019). To address this limitation and assist with future replications, additional analyses were conducted to provide a more technological description of progressive DTT as it was used within this study. Specifically, a trial-by-trial analysis occurred for every progressive DTT session. This included measuring how many times each stimulus was presented as the sample and as the comparison, how many trials the interventionist used instructive feedback (Nottingham et al., 2020), and which, if any, stimuli were present in the instructive feedback. Additionally, the type and frequency of trial used by the interventionist during progressive DTT teaching sessions was analyzed. Trial types were scored as either a matching trial, expressive trial, or receptive trial (described later). The data sheets used to conduct a trial-by-trial analysis for every progressive DTT teaching session is provided as a supplementary file.
Design and Procedure
An adapted alternating treatments design (Sindelar et al., 1985) was used to assess the effectiveness of each teaching condition. Participants progressed through this sequence of conditions: pretest, EBI and progressive DTT teaching sessions, daily probes, posttest, and maintenance. The mastery criterion was 100% correct responding across three consecutive daily probes. Probes and teaching sessions only occurred once per day and between 3 to 5 days a week depending on participant schedules.
Pretest and Posttest Probes
One pretest probe occurred prior to intervention, and two posttest probes occurred once participants met the mastery criterion for each training set. One posttest probe was conducted on the day the participant met the mastery criterion and the second posttest occurred on the next available day a participant was accessible for a research session. The time between the first and second posttest ranged from 1 to 7 days. Pretest and posttest probes consisted of five-trial blocks (i.e., 30 trials total) that included all relations across the stimulus classes (i.e., AB, BC, CA, AC, BA, and CB relations). To increase array size, two distractor stimuli were included in the array (i.e., five total stimuli). A large array decreased the likelihood of participants responding correctly by chance. Using distractors in an array has also been found to be more effortful for participants but does not impact learning of the target stimuli (Nussenbaum et al., 2017). Distractor stimulus classes were never targeted during teaching but were present during probes (see Table 3 for distractor stimuli in each set). When one of the stimuli from a distractor stimulus class served as the sample in a trial block during pretest and posttest probes, participant responding did not count toward the percentage of correct responding on the probes. Pretest and posttest probes were conducted similarly to Stanley et al. (2018) with each target and distractor stimulus presented once as the sample. After the sample was presented and the participant engaged in the observing response, the five comparison stimuli were presented. During pretest and posttest probes, the participants were given 15 s to respond and neutral feedback (e.g., “Okay,” “Thanks,” “Alright”) was provided by the interventionist regardless of participant responding. The location of the stimuli in the comparison array was randomized and the presentation sequence of the trial blocks was also randomized.
Table 3.
Adult Participant Distractor Targets
Daily Probes
Daily probes consisted of two trial blocks (i.e., 10 total trials) that assessed responding on AB and BC relations. A daily probe was conducted for each training set while intervention was occurring for each teaching condition. Similar to pretest and posttest probes, distractor stimuli were present in the comparison array and also served as samples for two of the trials in the trial block. These trials did not count toward the percentage correct on daily probes. After the sample was presented and the participant engaged in the observing response, the comparison stimuli were presented, and the participant was given 15 s to respond. Neutral feedback (e.g., okay, alright, thanks) was provided regardless of participant responding. The location of the stimuli in the comparison array was randomized and the order of trial blocks and trials within the block were randomized prior to each session. Daily probes always occurred prior to the teaching conditions and only one daily probe was conducted per day. Time between daily probes and teaching conditions ranged from 5 min to 3 hr depending on each participant’s schedule. The mastery criterion was 100% correct responding on three consecutive daily probes. After the participant met the mastery criterion, a posttest occurred.
Equivalence-Based Instruction
A linear-series training structure was used to teach AB and CB relations in the EBI teaching condition. The sample (e.g., picture of Vera Rubin) was presented, the participant was required to engage in an observing response (i.e., lifting the black square to reveal sample), the interventionist then turned the page to reveal the comparison array (e.g., Vera Rubin, Emmy Noether, Nettie Stevens), issued the instruction “Find the same,” and gave the participant 15 s to respond. If the participant selected the correct stimulus in the comparison array, the interventionist provided praise (e.g., “Yeah! You got it!”). If the participant did not select the correct stimulus in the comparison array, the interventionist provided corrective feedback and modeled the correct response (e.g., “No, it’s this one” while pointing to the name Vera Rubin). Each A stimulus was presented once as the sample with the B stimuli serving as the comparison array (i.e., three total trials), and each B stimulus was presented once as the sample with the C stimuli serving as the comparison array (i.e., three total trials). Unlike the pretest and posttest probes and daily probes, the distractor stimuli were not presented as the sample or in the comparison array during teaching. The location of the stimuli in the comparison array was randomized, and the order of trial blocks and trials within the block were randomized prior to each session. One teaching session occurred per day and continued until the participant reached the mastery criterion.
Progressive DTT
In the progressive DTT condition, the interventionist was not constrained to only teach certain relations nor to use a specific teaching method. The interventionist could target any relations in the training set, use any prompt type, error correction procedure, instructive feedback, different types of teaching trials (e.g., receptive, expressive, matching), and could choose the stimuli used as the sample and in the comparison array. The interventionist followed guidelines suggested by Leaf et al. (2016) including: (a) selecting trial targets and placement of stimuli based on learner responding within and across trials, (b) using natural language during instructions, (c) varying instructions, (d) using flexible prompt fading, (e) using instructive feedback in consequences to teach additional skills, and (f) using error correction when needed. The only constraint in this condition was that only 6 discrete trials could occur per teaching session. This was done to keep the number of teaching trials consistent across the two conditions.
Maintenance
Maintenance consisted of another pretest/posttest probe testing for all relations in the training set. The maintenance probe occurred 2 weeks after the mastery criterion was reached for each teaching condition.
Interobserver Agreement and Treatment Fidelity
A second independent observer recorded participant responding from video recordings of the sessions on pre/post probes, maintenance probes, and daily probes. Additionally, interobserver agreement (IOA) was calculated for variables assessed during DTT teaching sessions. IOA was calculated by taking the number of agreements trial-by-trial and dividing by the total number of trials and multiplying by 100. IOA was collected for 45.8% of pretest/posttest probe sessions, 50% of maintenance sessions, 37% of daily probes, and 36.4% of DTT teaching sessions. IOA for pretest/posttest probe sessions was 100%, 100% for maintenance sessions, 100% for daily probes, and 100% for DTT teaching sessions.
A second independent observer also recorded the interventionist’s implementation of pretest/posttest probes, maintenance probes, daily probes, DTT teaching sessions, and EBI teaching sessions to assess treatment fidelity. For pretest/posttest probes, maintenance probes, and daily probes the independent observer collected data on the interventionist: (a) presenting the page with sample, (b) requiring the participant engage in the observing response, (c) presenting the page with comparisons, (d) providing instruction (i.e., “find the same”), and (e) providing neutral feedback for every trial in the session. For DTT teaching sessions data were collected on the interventionist (a) providing an instruction, (b) providing a prompt after the instruction (if applicable), (c) providing 10 s for the participant to respond, (d) providing appropriate consequence, and (e) only conducting six trials per session. For EBI teaching sessions data were collected on the interventionist: (a) presenting the page with the sample, (b) requiring the participant engage in the observing response, (c) presenting the page with the comparisons, (d) providing the instruction (i.e., “find the same”), and (e) providing the appropriate consequence based on participant responding. Treatment fidelity was calculated by taking the number of trials in which the interventionist implemented all steps correctly and dividing by the total number of trials per session and multiplying by 100. Treatment fidelity was taken for 25% of pretest/posttest probe sessions, 25% of maintenance sessions, 28.4% of daily probe sessions, 36.4% of DTT teaching sessions, and 28.8% of EBI teaching sessions. Overall treatment fidelity for pretest/posttest probes was 99.5%, 100% for maintenance probes, 100% for daily probes, 100% for DTT teaching sessions, and 100% for EBI teaching sessions.
Results
Responding During Probes
Figure 1 displays the percentage of correct responding on preprobes/postprobes, daily probes, and maintenance probes for each participant.
Fig. 1.
Percentage of correct responding on probes across conditions for adult participants
Ann
Ann responded 27.7% and 22.2% correct on the preprobe for the stimulus sets assigned to the EBI and progressive DTT conditions, respectively. Ann met the mastery criterion for the stimulus set assigned to the progressive DTT and EBI conditions in four and nine teaching sessions, respectively. During posttests probes for all relations, Ann responded 100% correct for the stimulus sets assigned in both conditions. Ann responded correctly on 94.4% of maintenance trials in both conditions.
Donna
Donna responded 22.2% and 0% correct on the preprobe for the stimulus sets assigned to the EBI and progressive DTT conditions, respectively. Donna met the mastery criterion for the stimulus set assigned to the progressive DTT and EBI conditions in seven and 17 teaching sessions, respectively. During posttests probes for all relations, Donna responded 94.4% and 100% correctly in the progressive DTT condition and 100% correctly on both postprobes in the EBI condition. Donna responded correctly on 100% and 94.4% of maintenance trials in the progressive DTT and EBI condition, respectively.
April
April responded 0%, and 22.2% correct on the preprobe for the stimulus sets assigned to the EBI and progressive DTT conditions, respectively. April met the mastery criterion for the stimulus set assigned to the progressive DTT and EBI conditions in five and 10 teaching sessions, respectively. During posttests probes for all relations, April responded 100% and 94.4% correctly in the progressive DTT condition and 100% correctly on both postprobes in the EBI condition. April responded correctly on 100% of maintenance trials in both conditions.
Leslie
Leslie responded 11.1% and 22.2% correct on the preprobe for the stimulus sets assigned to the EBI and progressive DTT conditions, respectively. Leslie met the mastery criterion for the stimulus set assigned to the progressive DTT and EBI conditions in six and 23 teaching sessions, respectively. During posttests probes for all relations, Leslie responded 100% correct for the stimulus sets assigned in both conditions. Leslie responded correctly on 94.4% and 88.9% of maintenance trials in the progressive DTT and EBI condition, respectively.
Efficiency
Table 4 displays the efficiency of each condition as measured by teaching sessions and teaching time required to reach the mastery criterion for each participant. For all participants, the progressive DTT condition was more efficient than the EBI condition.
Table 4.
Adult Participant Efficiency Results
| Participant | DTT Condition | EBI Condition | ||
|---|---|---|---|---|
| Sessions to Mastery | Time to Mastery | Sessions to Mastery | Time to Mastery | |
| Ann | 4 | 4 min 01 s | 9 | 7 min 53 s |
| Donna | 7 | 5 min 32 s | 17 | 10 min 54 s |
| April | 5 | 5 min 07 s | 10 | 7 min 43 s |
| Leslie | 6 | 5 min 30 s | 23 | 18 min 18 s |
Social Validity
Table 5 displays participant responses to the social validity questionnaire. Overall, participants responded more favorably to the progressive DTT condition and found the teaching method to be more preferred, effective, and more appropriate in terms of teaching time. Participants rated the EBI condition less favorably and found the teaching method to be less preferred, less effective, and less appropriate in terms of teaching time. For the importance of the targets learned, most participants found the targets to be somewhat important, with April indicating that the targets were not at all important to learn. When asked to select a teaching method they would prefer to learn future targets in, all participants selected the DTT condition. Similarly, when asked to select a teaching method they would prefer to teach others with, all participants selected the DTT condition.
Table 5.
Adult Participant Responses to Social Validity Questionnaire
| Question | Ann | Donna | April | Leslie |
|---|---|---|---|---|
| 1. Rate the degree to which you would prefer to be taught using the discrete trial teaching method (i.e., “zot” condition). | 5 | 5 | 5 | 5 |
| 2. Rate the degree to which you would prefer to be taught using the equivalence-based instruction method (i.e., “wob” condition). | 2 | 1 | 1 | 1 |
| 3. How do you feel about the length of time it took to learn the targets in the discrete trial teaching method (i.e., “zot” condition). | 4 | 5 | 5 | 5 |
| 4. How do you feel about the length of time it took to learn the targets in the equivalence-based instruction method (i.e., “wob” condition). | 2 | 3 | 1 | 1 |
| 5. Rate the degree to which you thought the discrete trial teaching method (i.e., “zot” condition) was effective in teaching the targets. | 5 | 5 | 5 | 4 |
| 6. Rate the degree to which you thought the equivalence-based instruction teaching method (i.e., “wob” condition) was effective in teaching the targets. | 3 | 3 | 2 | 2 |
| 7. Rate the degree to which you thought the targets were important to learn. | 3 | 4 | 1 | 4 |
| 8. Which method would you prefer to learn future targets in? | DTT | DTT | DTT | DTT |
| 9. Which method would you prefer to teach targets to others in? | DTT | DTT | DTT | DTT |
Progressive DTT Analysis
For the progressive DTT teaching sessions, Table 6 displays how many times each stimulus was presented as the sample and as a comparison across all participants. All relations were targeted between 2 to 3 times, 1 to 7 times, 2 to 4 times, and 2 to 5 times for Ann, Donna, April, and Leslie, respectively. The interventionist also made frequent use of instructive feedback. Specifically, instructive feedback was used on 87.5%, 78.6%, 83.3%, and 80% of trials for Ann, Donna, April, and Leslie, respectively.
Table 6.
Total Number of Each Trial Combination for Adult Participant
| Sample | Comparison | Participant | |||
|---|---|---|---|---|---|
| Ann | Donna | April | Leslie | ||
| A1 | B1 | 2 | 5 | 4 | 3 |
| A2 | B2 | 2 | 4 | 4 | 3 |
| A3 | B3 | 2 | 6 | 4 | 3 |
| B1 | C1 | 2 | 2 | 2 | 2 |
| B2 | C2 | 2 | 1 | 2 | 3 |
| B3 | C3 | 2 | 2 | 2 | 2 |
| B1 | A1 | 2 | 7 | 2 | 3 |
| B2 | A2 | 2 | 2 | 2 | 3 |
| B3 | A3 | 2 | 4 | 2 | 3 |
| C1 | B1 | 2 | 2 | 3 | 5 |
| C2 | B2 | 2 | 2 | 3 | 4 |
| C3 | B3 | 2 | 2 | 3 | 5 |
| A1 | C1 | 3 | 3 | 4 | 2 |
| A2 | C2 | 3 | 3 | 4 | 2 |
| A3 | C3 | 3 | 3 | 4 | 2 |
| C1 | A1 | 2 | 2 | 3 | 5 |
| C2 | A2 | 2 | 3 | 3 | 5 |
| C3 | A3 | 2 | 3 | 3 | 5 |
Table 7 displays how frequently a stimulus from each stimulus class was included in the instructive feedback across each participant. While this frequency varied across participants, all stimuli in each stimulus class were included in instructive feedback at least three times.
Table 7.
Frequency of Stimuli Stated in Instructive Feedback Across Adult Participants
| Stimulus | Participants | |||
|---|---|---|---|---|
| Ann | Donna | April | Leslie | |
| A1 | 3 | 4 | 7 | 9 |
| A2 | 4 | 3 | 8 | 8 |
| A3 | 4 | 4 | 6 | 9 |
| B1 | 6 | 11 | 7 | 9 |
| B2 | 6 | 9 | 8 | 8 |
| B3 | 6 | 11 | 8 | 9 |
| C1 | 7 | 12 | 7 | 9 |
| C2 | 7 | 9 | 9 | 8 |
| C3 | 7 | 12 | 7 | 8 |
Table 8 displays the trial type and frequency used by the interventionist across participants. Matching trials were used most frequently followed by expressive trials. Receptive trials were not used with any participant.
Table 8.
Trial Type and Frequency Across Adult Participants
| Trial Type | Participants | |||
|---|---|---|---|---|
| Ann | Donna | April | Leslie | |
| Matching | 18 | 24 | 24 | 30 |
| Expressive | 6 | 18 | 6 | 6 |
| Receptive | 0 | 0 | 0 | 0 |
Discussion
The purpose of Experiment 1 was to compare progressive DTT with EBI with four neurotypical adults. Both instructional approaches resulted in mastery of the relations tested. This is not a surprising result given that the effectiveness of progressive DTT and EBI have been well documented. What is surprising is that all four participants reached mastery within the progressive DTT condition several sessions prior to doing so in the EBI condition, resulting in the progressive DTT condition being far more efficient for these participants. This preliminary data is surprising given the suggested efficient, economical possibility of EBI. Furthermore, all participants judged the progressive DTT condition to be more preferred, effective, and appropriate in terms of teaching time. The social validity results should be taken with caution due to the participants’ history of implementing behavioral intervention and training in the implementation of progressive DTT procedures. Although it is important to ask questions to participants regarding the social acceptability of the goals, outcomes, and procedures of an intervention (Wolf, 1978), these are also subjective self-report measures in which a participant’s history is an important variable that effects how they respond to the questions posed. Unlike typically developing adults that are unlikely to encounter learning through discrete trial procedures, children diagnosed with ASD, on the other hand, may frequently contact instruction through the use of progressive DTT as well as EBI.
Experiment 2: Children Diagnosed with ASD
Methods
The methods for Experiment 2 were identical to those in Experiment 1, with some exceptions outlined below.
Participants
The participants in Experiment 2 included four children diagnosed with ASD. Table 9 provides the demographic information each participant. Scores for IQ on the Wechsler Intelligence Scale for Children (WISC-V; Wechsler, 2012), Expressive One Word Picture Vocabulary Test (EOWPVT-4; Martin & Brownell, 2011), the Peabody Picture Vocabulary Test (PPVT-5; Dunn, 2019) were obtained 1 to 9 months prior to the onset of the study. Three of the participants were currently clients receiving services at the clinic where the study took place (i.e., Tammy, Ben, Tom) and one participant that was recruited from the community to participate in the study (i.e., Ron). Signed informed consent was obtained from the parents of each participant.
Table 9.
Child Participant Demographic Information
| Name | Age | Sex | Race/Ethnicity | Diagnosis | Education | IQ Score | EOWPVT | PPVT |
|---|---|---|---|---|---|---|---|---|
| Ron | 12 years | Male | Filipino/White | ASD | 6th grade | 91 | 119 | 111 |
| Tammy | 10 years | Female | White | ASD | 5th grade | 115 | 139 | 143 |
| Tom | 8 years | Male | Chinese-American/White | ASD | 3rd grade | 122 | 115 | 122 |
| Ben | 7 years | Male | White | ASD | 2nd grade | 132 | 130 | 130 |
Assessment scores for IQ, EOWPVT, and PPVT are all standard scores that have a mean of 100.
Interventionist
The first author continued to serve as the interventionist for Experiment 2.
Setting
All sessions took place in the same location as Experiment 1.
Targets and Materials
Two sets of three 3-member stimulus classes were taught to each participant. Similar to Experiment 1, one set was taught using EBI and the other set was taught using progressive DTT. Given the participant demographic (i.e., school-aged children), targets were selected that the participants were likely to contact within the curriculum of general education classrooms. Furthermore, similar stimuli have been used as targets within the equivalence literature (e.g., Ma et al., 2016). The stimulus classes consisted of U.S. state names, the corresponding state flags, and state flowers (see Table 10). Distractor stimuli present during daily probes and pretest/posttest probes can be found in Table 11. The stimuli and materials were arranged similarly to Experiment 1, with the exception of the state flag stimuli, which were printed 3-in. × 4-in. due to the flags’ shape. The state flag stimuli were printed 2-in. × 3in. on the pages with the comparison array. Training sets were counterbalanced across participants. For Ron and Tammy, set one was assigned to the progressive DTT condition and set two was assigned to the EBI condition. For Tom and Ben, set one was assigned to the EBI condition and set two was assigned to the progressive DTT condition.
Table 10.
Child Participant Targets
Table 11.
Child Participant Distractor Targets
Dependent Variables
The dependent variables for Experiment 2 were the same as Experiment 1, which included the percentage of correct responding on pretest/posttest probes, daily probes, and maintenance probes.
Ancillary Measures
Efficiency
Efficiency of the teaching procedures for Experiment 2 was evaluated in the same manner as Experiment 1.
Social Validity
Participant preference for each teaching condition within Experiment 2 was assessed using a modified concurrent chains arrangement (Hanley, 2010). Prior to the experiment, two preference assessments were conducted to select two neutral colors to represent each condition. Two paired stimulus preference assessments (Fisher et al., 1992) with 10 different colored sheets of paper were conducted with each participant. The results of the paired-stimulus preference assessments were used to select two neutral colors (i.e., not found to be highly preferred or nonpreferred) to represent the teaching conditions. After the two neutral colors were selected, they were randomly assigned to either the EBI teaching condition or the progressive DTT condition. The colored sheet of paper assigned to the designated teaching condition was present during each teaching session and on every third teaching session the child participant had a choice of which condition to start with that day (e.g., “Which one do you want to start with today, orange or pink?”). The condition the participant selected differentially was assumed to be the most preferred (e.g., Geiger et al., 2010; Hanley, 2010; Toussaint et al., 2016). Additionally, the participants were asked at the conclusion of the study which condition they liked the best (e.g., “If I was going to teach you more states, state flags, and state flowers which way would you rather learn? Orange or pink?”).
To further assess social validity, the parents of the child participants were sent a questionnaire with four questions at the conclusion of the study. All questions used a Likert-scale from 1 (i.e., Don’t prefer at all, Not at all important) to 5 (i.e., Strongly prefer, Very important) and below each question was space to write additional comments. To answer the questions, the parents were provided with video clips of a teaching session from each condition that were selected at random. Parents were provided with this written instruction that was at the top of the questionnaire: “Please watch the two videos we have provided. They are labeled ‘Clip 1’ and ‘Clip 2’. You can watch the videos as many times as you would like. Once you have watched both videos please answer the questions below. When completed, please email the completed form to [name] at [email] who will then give it to the research office anonymously.” The video clip labeled “Clip 1” was an EBI teaching session and “Clip 2” was a progressive DTT teaching session.
Progressive DTT Teaching session analysis
The same analysis from Experiment 1 occurred for the progressive DTT teaching sessions within Experiment 2.
Design and Procedure
Similar to Experiment 1, an adapted alternating treatments design (Sindelar et al., 1985) was used to assess the effectiveness of each teaching condition, progressing through the same sequence of conditions. The mastery criterion was set at 100% correct responding across three consecutive daily probes. Probes and teaching sessions only occurred once per day and between 3 to 5 days a week depending on participant schedules. All probe and teaching conditions were identical to those employed in Experiment 1.
Interobserver Agreement and Treatment Fidelity
A second independent observer recorded participant responding from video recorded sessions on preprobes/postprobes, maintenance probes, and daily probes. Additionally, interobserver agreement (IOA) was calculated for variables assessed during DTT teaching sessions. IOA was calculated by taking the number of agreements trial-by-trial and dividing by the total number of trials and multiplying by 100. IOA was collected for 41.7% of pretest/posttest probe sessions, 50% of maintenance sessions, 34.4% of daily probes, and 32.4% of DTT teaching sessions. IOA for pretest/posttest probe sessions was 99.3%, 100% for maintenance sessions, 100% for daily probes, and 98.9% for DTT teaching sessions.
A second independent observer also recorded the interventionist’s implementation of pretest/posttest probes, maintenance probes, daily probes, DTT teaching sessions, and EBI teaching sessions to assess treatment fidelity. The independent observer collected data on the same interventionist behavior as in Experiment 1. Treatment fidelity was taken for 25% of pretest/posttest probe sessions, 25% of maintenance sessions, 28.1% of daily probe sessions, 32.4% of DTT teaching sessions, and 27.1% of EBI teaching sessions. Overall treatment fidelity for pretest/posttest probes was 100%, 100% for maintenance probes, 99.9% for daily probes, 100% for DTT teaching sessions, and 100% for EBI teaching sessions.
Results
Responding During Probes
Figure 2 displays the percentage of correct responding on preprobes/postprobes, daily probes, and maintenance probes across each participant.
Fig. 2.
Percentage of correct responding on probes across conditions for child participants
Ron
Ron responded 22.2% correct on preprobes for both conditions. Ron met the mastery criterion for the stimulus set assigned to the progressive DTT and EBI conditions in six and 12 teaching sessions, respectively. During posttests probes for all relations, Ron responded 100% and 94.4% correct in both conditions. Ron responded correctly on 94.4% and 100% of maintenance trials for the DTT and EBI conditions, respectively.
Tammy
Tammy responded 33.3% and 16.7% correct on the preprobe for the stimulus sets assigned to the EBI and progressive DTT conditions, respectively. Tammy met the mastery criterion for the stimulus set assigned to the progressive DTT and EBI conditions in seven and 18 teaching sessions, respectively. During posttests probes for all relations, Tammy responded 100% correctly on both post-probes in the progressive DTT condition and 72.2% and 67.7% correct for the EBI condition. Tammy responded correctly on 100% and 72.2% of maintenance trials in the progressive DTT and EBI condition, respectively.
Tom
Tom responded 16.67% and 5.5% correct on preprobes for the stimulus sets assigned to the EBI and progressive DTT conditions, respectively. Tom met the mastery criterion for the stimulus set assigned to the progressive DTT and EBI conditions in nine and 14 teaching sessions, respectively. During posttests probes for all relations, Tom responded 94.4% correctly on both posttest probes in the progressive DTT condition and 83.3% and 88.9% correctly on post-probes in the EBI condition. Tom responded correctly on 88.9% and 100% of maintenance trials in the progressive DTT and EBI condition, respectively.
Ben
Ben responded 16.7% correct on the preprobe for the stimulus sets assigned to both conditions. Ben met the mastery criterion for the stimulus sets assigned to both conditions in 15 sessions. During posttests probes for all relations, Ben responded 100% correctly on both posttest probes in the progressive DTT condition and 83.3% and 88.9% correctly on postprobes in the EBI condition. Ben responded correctly on 94.4% and 100% of maintenance trials in the progressive DTT and EBI condition, respectively.
Efficiency
Table 12 displays the efficiency of each condition as measured by teaching sessions and teaching time required to reach the mastery criterion for each participant. The progressive DTT condition was more efficient with respect to sessions to mastery for Ron, Tammy, and Tom, while both conditions were equally efficient for Ben. In terms of teaching time to mastery, the progressive DTT condition was more efficient for Ron and Tammy and the EBI condition was more efficient for Tom and Ben.
Table 12.
Child Participant Efficiency Results
| Participant | DTT Condition | EBI Condition | ||
|---|---|---|---|---|
| Sessions to Mastery | Time to Mastery | Sessions to Mastery | Time to Mastery | |
| Ron | 6 | 6 min 37 s | 12 | 8 min 16 s |
| Tammy | 7 | 6 min 58 s | 18 | 12 min 29 s |
| Tom | 9 | 10 min 24 s | 14 | 8 min 46 s |
| Ben | 15 | 13 min 36 s | 15 | 9 min 53 s |
Social validity
Figure 3 displays the social validity results for child participants. The number of choices each participant was offered varied depending on how many teaching sessions were required for the participant to reach the mastery criterion for one or both conditions. Ron and Tammy selected each condition one time. Tom selected the progressive DTT condition twice and the EBI condition once. Ben selected the progressive DTT condition three times and the EBI condition once. At the conclusion of the study the participants were asked which condition they preferred, and all participants selected the DTT condition over the EBI teaching condition.
Fig. 3.
Number of selections between the DTT and EBI condition and overall selection made by child participants at the end of the study
Table 13 displays parent’s responses to the social validity questionnaire. Three of the four parents filled out and returned the questionnaire. Overall, parents responded more favorably to the progressive DTT condition (i.e., Clip 2) and found the teaching method to be more preferred. Parents rated the EBI condition (i.e., Clip 1) less favorably and found the teaching method to be less preferred. For the importance of the targets learned, parents found the targets to be somewhat important with ratings ranging from two to four. Although there was space on the questionnaire to write in additional comments none of the parents chose to write in additional information explaining their answer selections.
Table 13.
Parent Responses to Social Validity Questionnaire
| Question | Parent 1 | Parent 2 | Parent 3 |
|---|---|---|---|
| 1. Rate the degree to which you would prefer your child to be taught using the method in Clip 1. | 3 | 4 | 2 |
| 2. Rate the degree to which you would prefer your child to be taught using the method in Clip 2. | 5 | 5 | 4 |
| 3. How important was it for your child to be taught the targets/skills in Clip 1? | 3 | 4 | 2 |
| 4. How important was it for your child to be taught the targets/skills in Clip 2? | 3 | 4 | 4 |
Progressive DTT Analysis
Table 14 displays how many times each stimulus was presented as the sample and as the comparison for each participant in the progressive DTT condition. All relations were targeted between 1 to 6 times, 1 to 9 times, and 2 to 13 times for Ron, Tammy, and Ben, respectively. For Tom, BA relations were never targeted, but all other relations were targeted between 2 to 10 times. The interventionist also made frequent use of instructive feedback. Specifically, instructive feedback was used on 86.1%, 92.9%, 90.7%, and 74.4% of trials for Ron, Tammy, Tom, and Ben, respectively.
Table 14.
Total Number of Each Trial Combination for Each Child Participant
| Sample | Comparison | Participant | |||
|---|---|---|---|---|---|
| Ron | Tammy | Tom | Ben | ||
| A1 | B1 | 2 | 2 | 5 | 9 |
| A2 | B2 | 2 | 2 | 5 | 11 |
| A3 | B3 | 2 | 3 | 6 | 13 |
| B1 | C1 | 1 | 1 | 2 | 6 |
| B2 | C2 | 1 | 1 | 2 | 7 |
| B3 | C3 | 1 | 1 | 2 | 6 |
| B1 | A1 | 1 | 3 | 0 | 5 |
| B2 | A2 | 1 | 2 | 0 | 6 |
| B3 | A3 | 1 | 2 | 0 | 6 |
| C1 | B1 | 4 | 4 | 10 | 7 |
| C2 | B2 | 6 | 9 | 6 | 8 |
| C3 | B3 | 5 | 4 | 10 | 9 |
| A1 | C1 | 3 | 4 | 4 | 2 |
| A2 | C2 | 3 | 4 | 4 | 3 |
| A3 | C3 | 3 | 4 | 4 | 3 |
| C1 | A1 | 4 | 3 | 5 | 4 |
| C2 | A2 | 4 | 3 | 3 | 5 |
| C3 | A3 | 4 | 3 | 5 | 5 |
Table 15 displays how frequently a stimulus from each stimulus class was included in the instructive feedback across each participant. While this frequency varied across participants, all stimuli in each stimulus class were included in instructive feedback at least three times.
Table 15.
Frequency of Stimuli Stated in Instructive Feedback Across Child Participants
| Stimulus | Participants | |||
|---|---|---|---|---|
| Ron | Tammy | Tom | Ben | |
| A1 | 8 | 8 | 15 | 14 |
| A2 | 8 | 10 | 10 | 18 |
| A3 | 8 | 8 | 15 | 19 |
| B1 | 10 | 12 | 16 | 17 |
| B2 | 11 | 16 | 11 | 20 |
| B3 | 10 | 11 | 17 | 21 |
| C1 | 6 | 9 | 14 | 16 |
| C2 | 7 | 10 | 9 | 21 |
| C3 | 7 | 8 | 17 | 20 |
Table 16 displays the trial type and frequency used by the interventionist across participants. Matching trials were used most frequently followed by expressive trials, and receptive trials were only used for Ben.
Table 16.
Trial Type and Frequency Across Child Participants
| Trial Type | Participants | |||
|---|---|---|---|---|
| Ron | Tammy | Tom | Ben | |
| Matching | 24 | 33 | 31 | 43 |
| Expressive | 12 | 9 | 23 | 41 |
| Receptive | 0 | 0 | 0 | 6 |
Discussion
Experiment 2 was an attempt to replicate the results obtained in Experiment 1 with four individuals diagnosed with ASD. Similar to Experiment 1, both instructional approaches resulted in mastery of the relations tested. Results differed, however, with respect to efficiency measures. That is, for Ron, Tammy, and Tom the progressive DTT condition was more efficient with respect to sessions to mastery, while both conditions were equally efficient for Ben. Additional differences are found with respect to teaching time to mastery in which the progressive DTT condition was more efficient for Ron and Tammy and the EBI condition was more efficient for Tom and Ben.
In terms of the differences between teaching time to mastery versus sessions to mastery for Tom, this could be attributed to the structure of teaching sessions in the EBI and progressive DTT conditions. In the EBI condition, the intertrial interval remained relatively consistent and short because the learner’s behavior did not alter what occurred on the next trial. Regardless of correct or incorrect responding, the next page in the EBI binder was flipped over and the next trial began. Whereas in the progressive DTT condition, depending on the learner’s response on the previous trial this could alter what the interventionist did on the next trial. This likely extended the intertrial interval across the six trials in the teaching conditions in which the interventionist spent time assessing what should occur on the next trial, manipulating the comparison array, and deciding which stimulus to target on the next trial. For Ben, sessions to mastery were equivalent across the conditions, and the EBI condition was more efficient for time to mastery. Similarly to Tom, the longer time to mastery for the progressive DTT condition could have been due to longer intertrial intervals across trials during teaching. Although consistent across conditions, the use of extinction probe trials to determine mastery could explain the difference for Ben when it comes to the equal efficiency observed for sessions to mastery. In the progressive DTT condition, Ben came close to mastering the targets on Sessions 6 and 7 as well as on sessions 10 and 11 (i.e., two consecutive sessions at 100% correct responding). The teaching session data for Ben reflect that he was responding correctly in the absence of prompts across teaching trials, so the reinforcement contingency being absent from probe trials could have contributed to errors observed during daily probes. This information should be taken into account when choosing between procedures especially when time is more important than sessions, as EBI has consistently shorter intertrial intervals as compared with progressive DTT in which the intertrial intervals are likely to vary in terms of length.
When assessing preference for the two conditions, there were varied responses in the concurrent chain selections; however, all four participants selected the progressive DTT condition when asked which condition they preferred following completion of the study. These social validity results should be interpreted with caution due to the limited number of selections each participant had a chance to make. It should also be noted that the first selection in a concurrent chains arrangement is just one possible indicator of preference. However, there are several variables that could contribute to preference outside of order selection. For instance, some may choose the condition that is least preferred first in order to “get it out of the way,” so to speak. Nonetheless, other measures of social validity included in this study provide some level of validation to the concurrent chains data (e.g., cumulative number of selections, participant answer to final question of which one they preferred). Additionally, parents rated the progressive DTT condition more favorably than the EBI condition and found the targets taught to be moderately important. Similar to the questionnaire results obtained from the neurotypical adult participants, these responses to the social validity questionnaire may be influenced by the parent’s history observing behavioral intervention with their child. Parents may have rated the intervention that looked more familiar to them (i.e., DTT with laminated cards) higher than an intervention that looked unfamiliar (i.e., EBI done with a binder).
General discussion
To evaluate the effectiveness and efficiency of EBI, researchers have compared it to other instructional approaches (Ferman et al., 2020; Fienup & Critchfield, 2011; O’Neill et al., 2015). The two experiments described here represent the first comparison of EBI to a progressive approach to DTT. Experiments 1 and 2 yielded similar results to previous EBI research teaching academic targets in that all participants demonstrated the acquisition of derived relations after training in the EBI condition. All participants also demonstrated acquisition of the relations in the progressive DTT condition in which all relations were directly taught to participants with the exception of Tom who was never directly taught BA relations. The progressive DTT condition was found to be more efficient in terms of teaching time for 6 of the 8 participants while the EBI condition was found to be more efficient in terms of teaching time for two of the participants. When determining efficiency by sessions to mastery, the progressive DTT condition was found to more efficient for seven of the eight participants. All participants, across both experiments, responded similarly during the maintenance condition as they did on posttests for the EBI and progressive DTT conditions demonstrating that the relations learned maintained over time.
The social validity (Wolf, 1978) of the treatment goals, effects, and procedures was assessed in several ways. Adult participants were provided with a questionnaire about the importance of the goals, the acceptability of the treatment, and the treatment effects of each procedure. Overall, the adult participants rated the progressive DTT condition more favorably than the EBI condition. Higher ratings were reported for the length of time to learn the targets and the degree of effectiveness for teaching the targets in the progressive DTT condition. When assessing the importance of the targets, April responded not at all important, Ann responded somewhat important, and Donna and Leslie responded important.
Adult participant comments about the different teaching procedures may provide some more insight into why the progressive DTT condition was rated more favorably than the EBI condition. Ann wrote “The zot [progressive DTT] included thinking prompts which was extremely helpful (drawing attention to specific details of the person’s appearance or their contribution) as a way of remembering the correct pairing.” April wrote, “DTT seemed more natural as the other condition was so sterile and required more effort on my end to develop strategies to remember people/info.” Leslie wrote, “I felt [progressive DTT] was very effective, it helped for the instructor to say the name and provide a visual fact about the person’s appearance.” Taken together, it seems that the adult participants preferred the instructive feedback component used in the progressive DTT condition. When commenting on the EBI condition, Donna wrote, “I had to make my own connections/clues to learn the names and remember those which made me feel like I had to recall more information.” Ann also commented about the EBI condition stating, “It was effective but required my own extra strategies for learning the targets.” These comments regarding the EBI condition suggest the participants may have been engaging in covert behavior (e.g., instructive feedback similar to the instructive feedback the interventionist provided in the progressive DTT condition) that promoted learning the relations. Although these comments provide some insight they should be taken with caution due to each participant’s history and potential bias. These comments also shed some light into other differences between the two conditions that were not equivalent (e.g., instructive feedback, interventionist highlighting salient features of stimuli, sterility of EBI condition). It is possible that using another delivery method of EBI or using instructive feedback may increase the social acceptability of the procedure and should be evaluated in future research. Future research should also examine adult participant acquisition and social validity results with participants who frequently implement EBI procedures in order to assess if social validity results and acquisition of emergent and trained relations differ for adult participants with a history of EBI. Additionally, research should be conducted with adult participants naïve to both EBI and progressive DTT procedures in order to assess the effectiveness, efficiency, and social validity of the procedures in an absence of a history that could lead to biased responding.
Social validity was assessed through a concurrent schedule arrangement for the child participants in which each participant selected which condition they would like to contact first on every third session. Ron and Tammy selected each condition once, while Tom and Ben selected the progressive DTT condition more frequently. A limitation with this method was some participants were only provided with the choice twice before meeting the mastery criterion. Therefore, all participants were also asked to state their preference between the conditions, and all selected the DTT condition. Additionally, parents rated the DTT condition more favorably than the EBI condition and rated the targets taught as somewhat important.
Previous research on a progressive approach to DTT have cited limitations with respect to technological descriptions of the intervention due to the flexibility inherit to the intervention (e.g., assessing, responding, and changing interventionist behavior based on client behavior; Cihon et al., 2019; Leaf et al., 2018). The trial-by-trial analysis within this study represents a contribution to the literature in addressing that limitation. This analysis revealed that the interventionist directly trained all relations with the exception of one participant, Tom, in which the BA relations were never targeted. Instructive feedback was used on most trials for all participants. The differences between consequences provided in each condition and the frequent use of instructive feedback in the progressive DTT condition may explain why several participants learned the relations in the progressive DTT condition in fewer sessions than the EBI condition. Instructive feedback has been repeatedly demonstrated to result in the acquisition of nontarget information (Albarran & Sandbank, 2019). With respect to this study, instructive feedback commonly involved the targets in the stimulus classes which could have expedited learning. The use of instructive feedback did not occur at all in the EBI condition and instead only general praise for correct responses during teaching was provided (e.g., Yep! You got it!). These differences in terms of consequences may be a critical variable affecting efficiency and more research should be conducted to examine this variable. It is possible that the use of instructive feedback and variability in the consequences provided would increase the efficiency of the EBI condition for these participants or future participants in similar research.
Additionally, the incorporation of expressive and receptive trials in the progressive DTT condition constituted multiple exemplar instruction which has been shown to establish bidirectionality (LaFrance & Tarbox, 2020). Matching trials were used the most frequently and multiple exemplar instruction on these trails resulted in all participants engaging in tacts and/or echoics which has been demonstrated to result in untrained relations (LaFrance & Tarbox, 2020). Each of these components have demonstrated to lead to additional, untrained, and efficient learning in isolation, and this study demonstrated that learning can be accelerated when they are used in concert. Further, by selecting the stimuli included and the placement of those stimuli in the comparison array this permitted targeting two relations simultaneously on matching trials. For example, placing the A1 and B1, A2 and B2, and A3 and B3 stimuli in columns next to each other, then providing the participant the C2 stimulus with an instruction to match. These factors could also help explain the efficiency observed for six of the eight participants.
This study did not go without its limitations that warrant discussion. First, although we conducted a trial-by-trial analysis of all progressive DTT sessions, the variables that influenced the interventionist’s behavior (i.e., clinical judgment) during those sessions remain unclear. To address this limitation, future research should continue to expand by conducting a contingency analysis and component analysis to determine the conditions under which the interventionists or researchers respond to the learner in various ways. Future research could also have the interventionist review videotaped sessions and provide rationales for their behavior. Second, this study used a linear-series training structure within the EBI condition as well as instruction delivered using tabletop methods. It is possible that a different training structure (e.g., many to one, one to many; Arntzen & Holth, 2000) or modality would have yielded different results. Future studies should include comparisons of variations of EBI to progressive DTT to see if similar results are obtained. Third, the interventionist had an extensive history of providing instruction using a progressive approach to DTT within clinical and research settings (e.g., Cihon et al., 2019; Ferguson et al., 2020; Leaf et al., 2018). It is possible that EBI may be more efficient if staff are not as extensively trained in a progressive approach to DTT. While this may be a limitation when attempting to extend these results to the clinical setting, a tenet of a progressive approach to DTT is extensive training (Leaf et al., 2016). Nonetheless, future studies could evaluate effective training methods for staff to obtain similar results. Finally, this study made use of advances in our understanding and application of DTT, and it is possible that a combination of the two approaches could yield the most efficient and effective method of instruction. For example, thoughtfully selecting and targeting stimulus sets that lend themselves to class formation (like those used within this study), while employing multiple exemplar training, and instructive feedback are factors that may all combine to yield the most effective and efficient procedure. Additionally, research could be conducted on using progressive DTT embedded within an EBI intervention and conduct a component analysis to analyze the additive nature of either intervention.
With the importance of early intervention and the limited hours of intervention that some individuals diagnosed with ASD receive, the importance of efficient, economical, and socially valid interventions that maximize learning cannot be understated. While the efficient and economical argument for the use of EBI seems logical (i.e., the acquisition of untrained relations in less trials than training all relations directly), this study provides evidence that advances in DTT methodology may mitigate EBI’s efficiency over progressive DTT with data supporting progressive DTT to be more efficient in terms of sessions for seven out of the eight participants. Nonetheless, more research is warranted to replicate and extend these findings with a wider population and instructional variables.
Supplementary Information
(XLSX 401 kb)
(XLSX 730 kb)
(PDF 378 kb)
Compliance with ethical standards
Conflict of interest
The authors declare that there are no conflicts of interest.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the studies.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- Albarran SA, Sandbank MP. Teaching non-target information to children with disabilities: An examination of instructive feedback literature. Journal of Behavioral Education. 2019;28:107–140. doi: 10.1007/s10864-018-9301-3. [DOI] [Google Scholar]
- Arntzen E, Holth P. Equivalence outcome in single subjects as a function of training structure. The Psychological Record. 2000;50:603–628. doi: 10.1007/BF03395374. [DOI] [Google Scholar]
- Brodsky J, Fienup D. Sidman goes to college: A meta-analysis of equivalence-based instruction in higher education. Perspectives on Behavior Science. 2018;41(1):95–119. doi: 10.1007/s40614-018-0150-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cihon JH, Ferguson JL, Milne CM, Leaf JB, McEachin J, Leaf R. A preliminary evaluation of a token system with a flexible earning requirement. Behavior Analysis in Practice. 2019;12(3):548–556. doi: 10.1007/s40617-018-00316-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cihon JH, Ferguson JL, Leaf JB, Milne CM, Leaf R, McEachin J. A randomized clinical trial of three prompting systems to teach tact relations. Journal of Applied Behavior Analysis. 2020;53(2):727–743. doi: 10.1002/jaba.617. [DOI] [PubMed] [Google Scholar]
- Cooper JO, Heron TE, Heward WL. Applied behavior analysis. 3. Pearson; 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dixon MR, Belisle J, Stanley CR, Daar JH, Williams LA. Derived equivalence relations of geometry skills in students with autism: An application of the PEAK-E curriculum. The Analysis of Verbal Behavior. 2016;32:38–45. doi: 10.1007/s40616-016-0051-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dixon MR, Stanley C, Belisle J, Galliford ME, Alholail A, Schmick AM. Establishing derived equivalence relations of basic geography skills in children with autism. The Analysis of Verbal Behavior. 2017;33:290–295. doi: 10.1007/s40616-017-0084-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dunn DM. Peabody picture vocabulary test. 5. Pearson Assessments; 2019. [Google Scholar]
- Ferguson JL, Milne CM, Cihon JH, Dotson A, Leaf JB, McEachin J, Leaf R. An evaluation of estimation data collection to trial-by-trial data collection during discrete trial teaching. Behavioral Interventions. 2020;35(1):178–191. doi: 10.1002/bin.1705. [DOI] [Google Scholar]
- Ferman DM, Reeve KF, Vladescu JC, Albright LK, Jennings AM, Domanski C. Comparing stimulus equivalence-based instruction to a video lecture to increase religious literacy in middle-school children. Behavior Analysis in Practice. 2020;13:360–374. doi: 10.1007/s40617-019-00355-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fienup DM, Critchfield TS. Transportability of equivalence based programmed instruction: Efficacy and efficiency in a college classroom. Journal of Applied Behavior Analysis. 2011;44(3):435–450. doi: 10.1901/jaba.2011.44-435. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fienup DM, Covey DP, Critchfield TS. Teaching brain-behavior relations economically with stimulus equivalence technology. Journal of Applied Behavior Analysis. 2010;43(1):19–33. doi: 10.1901/jaba.2010.43-19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fisher W, Piazza CC, Bowman LG, Hagopian LP, Owens JC, Slevin I. A comparison of two approaches for identifying reinforcers for persons with severe and profound disabilities. Journal of Applied Behavior Analysis. 1992;25(2):491–498. doi: 10.1901/jaba.1992.25-491. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Geiger KB, LeBlanc LA, Dillon CM, Bates SL. An evaluation of preference for video and in vivo modeling. Journal of Applied Behavior Analysis. 2010;43(2):279–283. doi: 10.1901/jaba.2010.43-279. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ghezzi PM. Discrete trials teaching. Psychology in the Schools. 2007;44(7):667–679. doi: 10.1002/pits.20256. [DOI] [Google Scholar]
- Hanley GP. Toward effective and preferred programming: A case of the objective measurement of social validity with recipients of behavior-change programs. Behavior Analysis in Practice. 2010;3(1):13–21. doi: 10.1007/BF03391754. [DOI] [PMC free article] [PubMed] [Google Scholar]
- LaFrance DL, Tarbox J. The importance of multiple exemplar instruction in the establishment of novel verbal behavior. Journal of Applied Behavior Analysis. 2020;53(1):10–24. doi: 10.1002/jaba.611. [DOI] [PubMed] [Google Scholar]
- Leaf JB, Cihon JH, Leaf R, McEachin J, Taubman M. A progressive approach to discrete trial teaching: Some current guidelines. International Electronic Journal of Elementary Education. 2016;9(2):361–372. [Google Scholar]
- Leaf JB, Cihon JH, Ferguson JL, McEachin J, Leaf R, Taubman M. Evaluating three methods of stimulus rotation when teaching receptive labels. Behavior Analysis in Practice. 2018;11(4):334–349. doi: 10.1007/s40617-018-0249-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ma ML, Miguel CF, Jennings AM. Training intraverbal naming to establish equivalence class performances. Journal of the Experimental Analysis of Behavior. 2016;105(3):409–426. doi: 10.1002/jeab.203. [DOI] [PubMed] [Google Scholar]
- Martin NA, Brownell R. Expressive one-word picture vocabulary test. 4. Academic Therapy Publications; 2011. [Google Scholar]
- McLay LK, Sutherland D, Church J, Tyler-Merrick G. The formation of equivalence classes in individuals with autism spectrum disorder: A review of the literature. Research in Autism Spectrum Disorders. 2013;7(2):418–431. doi: 10.1016/j.rasd.2012.11.002. [DOI] [Google Scholar]
- Nottingham, C. L., Vladescu, J. C., DeBar, R. M., Deshais, M., & DeQuinzio, J. (2020). The influence of instructive feedback presentation schedule: A replication with children with autism spectrum disorder. Journal of Applied Behavior Analysis. Advanced online publication. 10.1002/jaba.706 [DOI] [PubMed]
- Nussenbaum K, Amso D, Markant J. When increasing distraction helps learning: Distractor number and content interact in their effects on memory. Attention, Perception, & Psychophysics. 2017;79:2606–2619. doi: 10.3758/s13414-017-1399-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- O’Neill J, Rehfeldt RA, Ninness C, Muñoz BE, Mellor J. Learning Skinner’s verbal operants: Comparing an online stimulus equivalence procedure to an assigned reading. The Analysis of Verbal Behavior. 2015;31(2):255–266. doi: 10.1007/s40616-015-0035-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rehfeldt RA. Toward a technology of derived stimulus relations: An analysis of articles published in the Journal of Applied Behavior Analysis, 1992–2009. Journal of Applied Behavior Analysis. 2011;44(1):109–119. doi: 10.1901/jaba.2011.44.109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sidman M. Reading and auditory-visual equivalences. Journal of Speech and Hearing Research. 1971;14(1):5–13. doi: 10.1044/jshr.1401.05. [DOI] [PubMed] [Google Scholar]
- Simon JL, Morris EK, Smith NG. Trends in women’s participation at the meetings of the Association for Behavior Analysis: 1975–2005. The Behavior Analyst Today. 2017;30(2):181–196. doi: 10.1007/bf03392154. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sindelar PT, Rosenberg MS, Wilson RJ. An adapted alternating treatments design for instructional research. Education and Treatment of Children. 1985;8(1):67–76. [Google Scholar]
- Soluaga D, Leaf JB, Taubman M, McEachin J, Leaf R. A comparison of flexible prompt fading and constant time delay for five children with autism. Research in Autism Spectrum Disorders. 2008;2(4):753–765. doi: 10.1016/j.rasd.2008.03.005. [DOI] [Google Scholar]
- Stanley CR, Belisle J, Dixon MR. Equivalence-based instruction of academic skills: Application to adolescents with autism. Journal of Applied Behavior Analysis. 2018;51(2):352–359. doi: 10.1002/jaba.446. [DOI] [PubMed] [Google Scholar]
- Toussaint KA, Kodak T, Vladescu JC. An evaluation of choice on instructional efficacy and individual preferences among children with autism. Journal of Applied Behavior Analysis. 2016;49(1):170–175. doi: 10.1002/jaba.263. [DOI] [PubMed] [Google Scholar]
- Wechsler D. Wechsler Preschool and Primary Scale of Intelligence–Technical manual and interpretive manual. 4. Pearson Assessments; 2012. [Google Scholar]
- Wesolowski MD. Pioneer profiles: An interview with Don Baer. The Behavior Analyst Today. 2002;25(2):135–150. doi: 10.1007/bf03392053. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wolf MM. Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis. 1978;11(2):203–214. doi: 10.1901/jaba.1978.11-203. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
(XLSX 401 kb)
(XLSX 730 kb)
(PDF 378 kb)







