Comparing Skill Acquisition Under Different Stimulus Set Sizes With Adolescents With Autism Spectrum Disorder: A Replication

Abstract

A number of variables may influence the effectiveness and efficiency of skill acquisition. One variable that may be important is set size. The current study replicated and extended Kodak et al. (2020, “A Comparison of Stimulus Set Size on Tact Training for Children With Autism Spectrum Disorder,” Journal of Applied Behavior Analysis, 53(1), 265–283) by evaluating the stimulus set size that led to the most efficient skill acquisition for 2 adolescents with autism spectrum disorder. More specifically, we evaluated tact acquisition in stimulus set sizes of 3, 6, and 12. The set sizes of 3 and 6 stimuli were associated with the most efficient acquisition, whereas the set size of 12 stimuli was not.

Discrete-trial instruction (DTI) is a frequently used procedure to establish new responses for learners with autism spectrum disorder (ASD). DTI involves training one or more unique responses, where the number of responses targeted simultaneously has been referred to as “stimulus set size” (Kodak et al., 2020). Practitioners and researchers implementing DTI may consider stimulus set size because this variable may influence the development of stimulus control (Kodak et al., 2020). Kodak et al. (2020) compared tact acquisition under stimulus set sizes of three, four, six, and twelve. Although all stimulus set size conditions were effective, the six- and twelve-stimuli conditions were most efficient across participants. One variable that may have influenced the results of Kodak et al. was that the mastery criterion employed required participants to emit fewer independent responses across stimuli as the set size increased. For example, in the four-stimuli condition, participants had to engage in six consecutive independent responses, whereas in the twelve-stimuli condition, participants had to engage in two consecutive independent responses to meet the mastery criterion. The purpose of the current study was to replicate and extend Kodak et al. by evaluating tact acquisition in stimulus set sizes of three, six, and twelve. In addition, we required the same number of independent responses across stimuli for all conditions, as opposed to Kodak et al., where the number of independent responses across stimuli differed across conditions.

Method

Participants

Two adolescents diagnosed with ASD participated in the study. Both participants received services based on the principles of applied behavior analysis and had recent histories of using a three-stimuli set size during DTI. Neither participant had a known history of using six- and twelve-stimuli set sizes during DTI. Lee, a 13-year-old male, scored a 37 (qualitative description: extremely low) on the Peabody Picture Vocabulary Test, Fourth Edition (PPVT-4; Dunn & Dunn, 2007), and a 32 (extremely low) on the Expressive Vocabulary Test, Second Edition (EVT-2; Williams, 2007). He received a total score of 121.5 on the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP; Sundberg, 2008). Dion, a 14-year-old male, scored a 26 on the PPVT-4 (extremely low) and a 23 (extremely low) on the EVT-2. He received a total score of 139 on the VB-MAPP.

Setting and Materials

The study was conducted in a room at a private school for learners with ASD. The room contained a table, two chairs, paper data sheets, pencils, stimuli, putative reinforcers, and two timers. Stimuli included pictures of household objects (e.g., washing machine, tape measure, envelope) that were presented on PowerPoint slides. The experimenter sat across the table from the participant.

Experimental Design, Measurement, Interobserver Agreement, and Procedural Integrity

We used an adapted alternating-treatment design (Sindelar, Rosenberg, & Wilson, 1985) to compare stimulus set size conditions. The experimenter conducted sessions across conditions in a randomized without-replacement fashion. Additionally, a multiple-baseline design across stimulus sets was used in the three- and six-stimuli conditions. Similar to Kodak et al. (2020), Fig. 1 depicts responding across conditions on different tiers to ease viewing of acquisition.

Fig. 1.

Percentage of unprompted correct responses. BL = baseline; EC = error correction

The experimenter recorded unprompted and prompted correct and incorrect responses during all sessions. We defined unprompted and prompted correct responses as the participant emitting the target response prior to or following the experimenter’s prompt, respectively. We defined unprompted and prompted incorrect responses as the participant engaging in an error of commission or omission prior to or following the experimenter’s prompt, respectively. Figure 1 depicts only the percentage of unprompted correct responses, which was calculated by dividing the total number of unprompted correct responses by the total number of responses in a session and multiplying that number by 100.

The experimenter recorded the total duration of each session using a digital timer by starting the timer immediately prior to establishing attending behavior prior to the first trial of the session, and stopping the timer following the terminal component of the last trial in the session. We calculated total training time by adding the session durations per condition.

A secondary observer independently scored participant responding and session duration from video for a minimum of 28% of sessions across conditions. Trial-by-trial interobserver agreement (IOA) was calculated by dividing the number of agreements by the number of agreements plus disagreements and converting that ratio to a percentage. An agreement was scored if the secondary observer recorded the same dependent variables as the primary observer within the trial. A disagreement was scored if the secondary observed recorded any different dependent variables from the primary observer within the trial. For session duration, we calculated total duration IOA by dividing the shorter duration by the longer duration and converting the number to a percentage. For Dion, the mean IOA score was 99.6% (range 92%–100%) and the mean duration IOA score was 99.5% (range 95%–100%). For Lee, the mean IOA score was 99.8% (range 92%–100%) and the mean duration IOA score was 99.5% (range 98%–100%).

An independent observer collected procedural integrity data for each participant for a minimum of 17% of sessions across all conditions. Procedural integrity was collected using a checklist (available from the first author) and was calculated by dividing the total number of correct experimenter behaviors by the total number of correct and incorrect experimenter behaviors. Mean procedural integrity scores for Dion and Lee were 99.7% (range 94%–100%) and 99% (range 94%–100%), respectively.

Preference Assessments

The experimenter conducted a multiple-stimulus without-replacement preference assessment (MSWO; DeLeon & Iwata, 1996) with each participant prior to the beginning of the evaluation. The experimenter conducted a brief MSWO (Carr, Nicolson, & Higbee, 2000) with five edibles prior to the start of each session. The top three edibles approached were delivered during the subsequent session as putative reinforcers. The edibles were cut into approximately equal bite-sized pieces.

Pretest and Stimuli Assignment

A pool of potential targets was selected based on individual treatment goals. The experimenter conducted pretest trials (details are available from the first author) and discarded any potential target to which the participant engaged in an unprompted correct response. Additionally, the experimenter conducted an echoic assessment (Carroll & Klatt, 2008) and discarded any potential target that a participant did not imitate or had difficulty imitating. We assigned twelve targets to each of the three conditions using a logical analysis (Wolery, Gast, & Ledford, 2014; a list of targets is available from the first author). The logical analysis included consideration of the following dimensions: number of syllables in each target name, redundancy of phonemes across target names, and physical similarity across targets.

General Procedure

All sessions consisted of twelve trials. The experimenter conducted three to seven sessions per day, 3 to 5 days per week.

Baseline

The experimenter established attending behavior (eye contact and hands in lap), presented a PowerPoint slide depicting an item, said “What is it?” and waited up to 5 s for the participant to respond. No feedback was provided for unprompted correct or incorrect responses. The experimenter presented the next trial after a 3- to 5-s intertrial interval (ITI). An edible and praise were provided for appropriate attending and sitting behaviors approximately every other trial during the ITI.

Training

Initially, the experimenter provided an echoic prompt (i.e., 0-s prompt delay [PD]) immediately following the presentation of the antecedent stimuli. Following a prompted correct response (0-s trials), the experimenter delivered an edible and praise. Following a prompted incorrect response (0-s trials), the experimenter presented the next trial. The experimenter increased the PD to 5 s after two consecutive sessions with 100% prompted correct responses at the 0-s PD. During trials conducted at the 5-s PD, the experimenter established attending behavior, presented the stimulus card, said the verbal antecedent stimulus, and waited up to 5 s for the participant to respond. Following unprompted correct responses (5-s trials), the experimenter delivered an edible and praise. Following an unprompted incorrect response (5-s trials), the experimenter provided an echoic prompt. Following a prompted incorrect response (5-s trials), the experimenter moved to the next trial. Following a prompted correct response (5-s trials), the experimenter delivered an edible and praise. The experimenter continued to deliver an edible and praise following prompted correct responses until the participant demonstrated unprompted correct responses during 50% or more of the trials. Thereafter, the experimenter delivered praise only following prompted correct responses. Unlike Kodak et al. (2020), we specified condition-specific mastery criteria to ensure that participants demonstrated similar unprompted correct responses across stimuli in all conditions. This was necessary because each stimulus was presented a different number of times in a session across conditions. Therefore, the experimenter continued training in the three-, six-, and twelve-stimuli conditions until the participant demonstrated unprompted correct responding during 100% of trials for one, two, and four sessions, respectively.

Three-stimuli set size

The experimenter divided twelve targets into four sets of three stimuli and sequentially trained each of the four sets of stimuli. During a training session, the experimenter presented each of the three stimuli during four trials (i.e., 3 stimuli multiplied by 4 trials equals 12 trials) randomly with the exception that the same target was not presented on more than two consecutive trials.

Six-stimuli set size

The experimenter divided twelve targets into two sets of six stimuli and sequentially trained each of the two sets of stimuli. During a training session, the experimenter presented each of the six stimuli during two trials (i.e., 6 stimuli multiplied by 2 trials equals 12 trials) randomly with the exception that the same target was not presented on more than two consecutive trials.

Twelve-stimuli set size

The experimenter presented twelve targets in one set of twelve stimuli and trained all targets simultaneously. During a training session, the experimenter presented each of the twelve stimuli during twelve trials randomly.

Results

During his comparison, Lee (Fig. 1, top) demonstrated mastery in the six-, three-, and twelve-stimuli conditions in 39 (81 min 40 s total training time), 49 (105 min 25 s total training time), and 59 (121 min total training time) training sessions, respectively. Dion (Fig. 1, bottom) demonstrated mastery in the three-stimuli condition in 116 training sessions (275 min 20 s total training time). Although Dion mastered Set 1 of the six-stimuli condition and had an increase in independent correct responding following the introduction of training of Set 2 of the six- and twelve-stimuli conditions, we observed stagnant responding. Therefore, the experimenter implemented a re-present until independent error correction procedure (see Carroll, Joachim, St. Peter, & Robinson, 2015, for a description) until mastery-level responding was achieved. In total, Dion demonstrated mastery in the twelve- and six- stimuli conditions in 157 (364 min 57 s total training time) and 168 (383 min 29 s total training time) training sessions, respectively.

Discussion

The aim of the current study was to systematically replicate and extend Kodak et al. (2020). The six-stimuli (Lee) and the three-stimuli (Dion) set sizes were found to be the most efficient, whereas the twelve-stimuli set size was not found to be the most efficient for either of the participants.

Our finding that the stimulus set size of three was most efficient for Dion differs from that of Kodak et al. (2020), who found the six- and twelve-stimuli set size conditions to be the most efficient. Our study modified the mastery criterion, in which the same number of unprompted responses to a target would result in mastery, whereas in Kodak et al., this dimension differed. For example, during the three-stimuli set size condition in the present study, mastery levels of responding were met after 100% unprompted correct responding in one session (i.e., four unprompted correct responses to a target were required for mastery), as opposed to Kodak et al., who required 100% unprompted correct responding for two consecutive sessions for the targets to be considered mastered (i.e., eight unprompted correct responses were required for mastery). Differences in results could also suggest that like other dimensions of DTI (e.g., differential reinforcement; Johnson, Vladescu, Kodak, & Sidener, 2017), the most efficient stimulus set size may be due to variables that differ with respect to participants. Moreover, the twelve-stimuli set size was never associated with optimal outcomes for either participant. That is, this condition required the most training sessions and the longest total training time across participants. This condition could have been the least efficient across participants due to the number of unique discriminations required during each session. The methodological differences in the current study and Kodak et al. suggest the need for future researchers to identify the conditions under which teaching should occur using specific set sizes. Additionally, consideration should be given to the mastery criterion employed when evaluating different set sizes.

When evaluating the data in the present study, it is important to consider that the participants were exposed to training of up to 21 targets simultaneously. That is, we conducted training across conditions simultaneously. This number of targets is significantly more than participants were exposed to during their typical programming, which may have influenced their responding in ways we were unable to account for. Future researchers might consider ways to evaluate whether this arrangement may promote or inhibit learning. Additionally, we conducted ongoing baseline probes for the three- and six-stimuli sets not yet exposed to training. This may have had a detrimental influence on participant responding if they continuously emitted a specific incorrect response during baseline that persisted following the introduction of training. Although an evaluation of this pattern is outside of the scope of this article, the conditions that involved an increased exposure to baseline (the three- and six-stimuli set size conditions) were the ones associated with the most optimal treatment efficiency data.

It is worth noting that we were unable to conduct intrasubject replications similar to Kodak et al. (2020). Given this limitation, our findings should be interpreted with appropriate caution, and future researchers should include replications to demonstrate the generality of outcomes. In addition, future researchers should aim to evaluate the generality of set size influence across other skill types. That is, Kodak et al. and the current study evaluated the influence of set size manipulations on participant acquisition of one type of simple discrimination (tacts), but it is unclear whether similar outcomes would have been observed if other types of simple discriminations were targeted. Moreover, how set size might influence the learning of other discriminations (e.g., conditional discriminations) is an important area for future researchers to consider.

Implications for Practice

1. Evaluates the relative effectiveness and efficiency of three stimulus set size conditions for the acquisition of tacts.

2. Demonstrates all stimulus set size conditions can be effective.

3. Demonstrates that the smaller set sizes were most efficient for both participants.

4. Provides a framework for practitioners to evaluate what stimulus set size will lead to more efficient skill acquisition.

Funding

This study received no funding.

Data availability

The data that support the findings of this study are openly available in figshare: 10.6084/m9.figshare.11396868.v1.

Compliance with Ethical Standards

Conflict of interest

All authors report no conflicts of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.