Referent-Based Instruction to Strengthen the Verbal Behavior of Early Learners with Autism and Related Language Disorders

Abstract

The current study evaluated the use of precision teaching to address the verbal behavior deficits of children with autism and other language disorders. From 2013 to 2018, a high-research-activity doctoral university in the south-central United States operated a free clinic that provided applied behavior anlaysis services to early learners in the local community. Participants received referent-based verbal behavior instruction to strengthen their functional language skills by systematically transferring stimulus control across 4 primary verbal operants: mands, echoics, tacts, and sequelics. Referent-based instruction is premised on the notion that proportionate levels of strength among these 4 operants provide the relational flexibility of naturalistic speaking observed in typical language development. This article details the language gains made by 49 participants who received 13 weeks of intervention for 90 min a day, 4 days a week. Relative strengths and weaknesses were identified in the verbal repertoire of each participant, and individualized fluency aims were subsequently developed. Results of pretest and posttest comparisons show that there was a large effect size within the verbal behavior gains of participants who received precision teaching. Implications for implementing referent-based instruction, as well as future areas of research, are discussed.

Language impairment is a core deficit of autism spectrum disorder (ASD). Wan et al. (2011) estimated that approximately one quarter of individuals with ASD are nonverbal. Of those who develop language, less than half attain fluency (Wodka, Mathy, & Kalb, 2013). Although early intensive behavioral intervention has been shown to remediate the language skills of individuals with ASD, the inability of some children to develop fluent speech may be a function of stimulus overselectivity (Brown & Bebko, 2012; Reed, Stahmer, Suhrheinrich, & Schreibman, 2013). That is, their verbal behavior has been adequately conditioned under some, but not all, relevant properties of the environment.

Imagine a child who walks into a toy store and sees a preferred item on the top shelf, said Skinner (1957, p. 188). “Dad, what’s that?” he queries. “That, my boy, is a doodler,” his father replies, providing the echoic stimulus that is commonly used to condition control of the tact. “Buy me that doodler!” the child spontaneously mands.

Of primary importance in this verbal episode is the ease with which a novel response is first acquired under one source of control, and then transferred to another: “He has never been reinforced for this response in the manner required to construct a mand” (Skinner, 1957, p. 188). Fluent speakers regularly demonstrate transfer of stimulus control within their daily verbal episodes. On the other hand, early language learners with ASD and related disorders are susceptible to sources of prepotent stimulus control that inhibit transfer (Mason, Davis, & Andrews, 2015). Such stimulus overselectivity may usurp other features of the environment from acquiring control.

The emergence of novel verbal operants first described by Skinner (1957) has been documented in the ensuing literature on verbal behavior, although the specific controlling variables that occasion emergence have yet to be identified (Cihon, 2007). Lamarre and Holland (1985) first documented the functional independence of mands and tacts by training nine neurotypical preschoolers to either mand or tact using simple prepositional statements (e.g., “On the left.”). Results showed that simply training targets within one operant class did not establish generalized control to collateral conditions. Whereas the majority of work on the emergence of untrained operant classes has focused on mand and tact transfer (Egan & Barnes-Holmes, 2009, 2011; Finn, Miguel, & Ahearn, 2012; Hall & Sundberg, 1987; Nuzzolo-Gomez & Greer, 2004; Petursdottir, Carr, & Michael, 2005; Pino, Leone, Forconi, & Casarini, 2010; Sigafoos, Reichle, Doss, Hall, & Pettitt, 1990; Twyman, 1996; Wallace, Iwata, & Hanley, 2006), other researchers have investigated collateral response acquisition across listener (Egan & Barnes-Holmes, 2010; Ribeiro, Elias, Goyos, & Miguel, 2010) and intraverbal relations (Miguel, Petursdottir, & Carr, 2005; Sundberg, San Juan, Dawdy, & Argüelles, 1990). Findings from these reports support the conclusion that enhanced instruction on individual operants appears to facilitate transfer to other operants.

Conversely, there is evidence to support the simultaneous training of multiple operants to strengthen responding within a single operant class. Carroll and Hesse (1987) found that typically developing preschoolers required fewer trials, on average, to condition responding in the presence of nonverbal stimuli using a mand–tact condition when compared to a tact-only condition. Similar results were found by both Arntzen and Almås (2002) and Luccherino and Scali (2013), who replicated Carroll and Hesse’s (1987) procedures using participants with developmental disabilities.

Kodak and Clements (2009) further extended this line of research by examining the effects of echoic prompts. Their participant, a 4-year-old boy with ASD, failed to acquire unprompted tacts and mands during single-operant training. However, the introduction of imitative verbal stimuli helped condition nonverbal and motivational control.

With independently verified studies to support both the training of a single verbal operant to facilitate the emergence of other verbal operants and the simultaneous training of multiple verbal operants to expedite the conditioning of each, it has become increasingly difficult to overlook the growing body of literature that supports the functional interdependence of verbal operants (Fryling, 2017). Consequently, behavior analysts have begun to question the pure independence of verbal operants (Bondy, Tincani, & Frost, 2004; Edwards, Lotfizadeh, & Poling, 2019; Mason & Andrews, 2019; Meindl, Miller, & Ivy, 2018; Michael, Palmer, & Sundberg, 2011). Edwards et al. (2019) specifically observed that to some degree the mand is always multiply controlled by the ongoing nonverbal or verbal stimulus context, whereas tacts only occur when the relevant social reinforcers are in effect. Conceptual distinctions aside, it is worth considering that if motivating operations “increase the evocative influence of relevant S^Ds [discriminative stimuli], we are directed to attend to these environmental stimuli, which reliably predict the occurrence of the behavior of interest” (Edwards et al., 2019, p. 4). Acknowledging the symbiosis of motivating operations and S^Ds may be useful for referencing the controlling relations that allow for the successful transfer of stimulus control within social interactions.

Children with fluent verbal repertoires, as described by Skinner (1957), demonstrate the relational flexibility to respond under unstable stimulus conditions (O’Toole & Barnes-Holmes, 2009; O’Toole, Barnes-Holmes, Murphy, O’Connor, & Barnes-Holmes, 2009). That is, fluent speakers must be able to maintain a consistent verbal behavior stream despite the ebb and flow of ever-changing environmental conditions. Palmer (2009) noted that “the apparent unity of emitted behavior masks a bedlam of concurrent fluctuations in strength of responses in the repertoire but below the threshold of emission” (p. 49). Whereas weak or otherwise faulty stimulus control allows for multiple, incompatible responses to reach a liminal intensity, fluent speakers demonstrate the speed and accuracy of relational flexibility.

Fluency measures are a fundamental component of precision teaching, a measure of conditioning precise stimulus control over a given response. Vargas (1977) observed that “teaching is not only producing new behavior, it is also changing the likelihood that a student will respond in a certain way. Since we cannot see a likelihood, we look instead at how frequently a student does something” (p. 62). In other words, establishing discriminative control through programs like precision teaching may be a useful instructional enhancement to address the “bedlam” of potentiation described by Palmer (2009).

Since its foundation in the 1970s, precision teaching has been used to address verbal behavior deficits, focusing primarily on those related to textual stimuli (Downs & Morin, 1990; Haughton, 1972; Hughes, Beverley, & Whitehead, 2007; Kubina & Starlin, 2003; Starlin, 1971). More recently, researchers have also demonstrated the effects of precision teaching for conditioning classes of other verbal operants, including mands (Solis, Derby, & McLaughlin, 2003), tacts (Ferris & Fabrizio, 2009), echoics (Foley & Fabrizio, 2005), and intraverbals (Cihon, 2007; Cihon et al., 2017; Emmick, Cihon, & Eshleman, 2010).

Precision teachers use fluency aims within learning channels as predetermined performance criteria for making data-based decisions. Learning channels can be described as a system of exteroceptive inputs and behavioral outputs for pinpointing behavior. Fluency aims are often presented as a range of frequencies designating a low end and a high end for performance (Binder, 1996; Haughton, 1980; Kubina & Wolfe, 2005). For example, Cihon et al. (2017) established a frequency aim of 80 to 90 words read per minute without errors. Fluent performance within this range is said to promote retention, endurance, application, and performance standards (Binder, 1996; Haughton, 1980; Lindsley, 1992).

Critiques of precision teaching have noted broad variability in its reported practice, along with seemingly arbitrary performance standards that are not individualized to the learner (Cihon, 2007; Doughty, Chase, & O’Shields, 2004; Heinicke, Carr, LeBlanc, & Severtson, 2010). For instance, the frequency aim established by Cihon et al. (2017) was based on suggestions from prior research (i.e., Emmick et al., 2010), or more general recommendations (i.e., Kubina, Morrison, & Lee, 2002). Moreover, Fabrizio and Moors (2003), argued that fluency aims cannot be established a priori, as teaching to a given aim has not been shown to be a reliable predictor of retention, endurance, stability, application, and adduction (Doughty et al., 2004; Heinicke et al., 2010).

To address the issue of standards and individualization, Mason and Andrews (2014) recommended looking across learning channels for performance aims that are already within the learner’s repertoire. For example, many children with ASD show disproportionate levels of stimulus control over their verbal behavior. A common language profile might show that the child can label and copy at twice the strength of requesting, along with little to no conversing. Mason and Andrews (2014) described a model of conditioning verbal behavior called referent-based instruction (RBI) that focuses on bringing each of four primary verbal operants under proportionate stimulus control.

RBI is a behavior-analytic treatment package that consists of verbal behavior training and precision teaching. What distinguishes RBI from other applied verbal behavior programs is its focus on the transfer of stimulus control across verbal operants. Mason and Andrews (2014) demonstrated the effects of RBI on a small sample of children, using the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP; Sundberg, 2008) as the primary outcome measure. The present study extends the utility of RBI by examining its effect on conditioning proportionate stimulus control over the verbal behavior of 49 children diagnosed with, or at risk of, ASD.

Method

Experimental Design

To minimize any potential selection bias favoring particular outcomes, we employed a consecutive case series analysis to measure the effects of RBI. Clinical data from the treatment records for all children who received behavior-analytic intervention through a university-based autism research center between January 2013 and May 2018 were reviewed and analyzed to examine the effects of RBI on language outcomes measured by the verbal behavior Stimulus Control Ratio Equation (SCoRE; Mason & Andrews, 2019). All cases for which both pre- and post-SCoRE data were collected met the following inclusion criteria: (a) the child had a pretest SCoRE of <.92, above which we determined the child’s speaking repertoire to be fluent; (b) there were no missing data for the duration of the child’s treatment; (c) sufficient interobserver agreement data were available; and (d) the child’s legally authorized representatives consented to the dissemination of aggregated data.

The verbal behavior SCoRE is an assessment for determining the proportionality of operants that compose the speaker’s repertoire. Specifically, the strength of mand, tact, and intraverbal—both echoic and sequelic—relations are examined. The SCoRE has been shown to have high construct validity with the VB-MAPP (Mason et al., 2018).

The verbal behavior SCoRE is a dynamic measurement tool in that the items to be assessed are identified by the participant through stimulus preference assessments. By employing a functional analysis of verbal behavior (Lerman et al., 2005), each verbal response is assessed across mand, echoic, tact, and sequelic control. The number of responses for each verbal operant is then calculated and compared against a model of proportionate response strength. The SCoRE yields a single numerical value, ranging between 0 and 1, that summarizes the speaking repertoire.

Consistent with Ferguson’s (2009) estimates for effect size, SCoRE results may be useful for describing the strength of the speaking repertoire. Below .20, the verbal repertoire can be described as “emergent.” Greater than or equal to .20, the repertoire is considered “practical.” A “moderate” repertoire refers to a SCoRE at or above .50. And with a SCoRE greater than or equal to .80, the speaker may be described as having a “strong” repertoire. This terminology allows for a precise discussion of a complex range of language profiles in plain English.

For the purposes of the current study, the SCoRE was used as a primary measure of functional language at both the onset and cessation of treatment. Trained graduate students, under the supervision of two Board Certified Behavior Analysts, conducted all SCoRE assessments. To assess interobserver agreement (IOA), two observers independently and simultaneously collected data for 82% of SCoRE assessments. Across participants, trial-by-trial IOA coefficients averaged 96% agreement (range 83%–100%).

Participants

We reviewed a total of 71 cases for this research. Strong verbal repertoires on the pretest SCoRE, missing or incomplete pretest and/or posttest assessments, attrition from the program, or an inability for us to acquire informed consent led to the elimination of 22 cases. This left 49 participants with ASD and related language disorders between the ages of 2 years, 0 months, and 6 years, 6 months. The average age of participants was 3 years, 5 months. Male and female participants came from a variety of culturally, socioeconomically, and linguistically diverse backgrounds. All participants demonstrated some sort of language impairment. The majority, 86%, were diagnosed with ASD. The remaining 14% were diagnosed with another language disorder. In addition to a primary diagnosis, 20% were also diagnosed with a comorbidity. All diagnoses were completed off-site; caregivers provided paperwork to document the need for functional language intervention services.

These participants took part in a university-based applied behavior analysis clinic over the course of 5 years. Behavior-analytic intervention was provided free of charge for the duration of one long semester, fall or spring. During the initial and final weeks of the intervention, we conducted a verbal behavior SCoRE, which served as a pre- and postmeasure of language performance. The results of each participant’s SCoRE assessment led to the development of an individualized treatment program in which the strongest verbal operant served as a benchmark for comparing the strength of the other operants.

Of the 49 participants who took part in this research, 84% had already acquired some degree of verbal behavior. Three participants spoke using augmentative and alternative communication, including sign language, picture exchange, and speech-generating devices. Five participants began with no functional speaking repertoire. Additionally, all participants engaged in some form of challenging behavior.

Based on their initial SCoRE results, we identified 11 participants with emergent verbal repertoires (<.20), 11 participants with practical verbal repertoires (.20–.49), 23 participants with moderate verbal repertoires (.50–.79), and 4 participants with strong verbal repertoires (≥.80) prior to treatment.

Setting

The procedures took place in an open clinical space, approximately 1,200 ft² (111 m²) in area. The room contained a number of age-appropriate toys, as well as structures for physical play (i.e., jungle gym, slide, trampoline, crawl tunnel, playhouse). On opposite ends of the room were a total of six workstations—three on each wall—which included a child-height rectangular table and two chairs. Wooden compartment shelves positioned perpendicular to the wall separated the workstations, serving the dual purposes of creating a space for table work and providing teachers with a place to store materials.

Undergraduate- and graduate-level university students studying educational applications of behavior analysis served as teachers. These students volunteered 25 hr per week of their time to accrue supervised field experience as part of an intensive practicum in early intensive behavioral intervention for ASD. Students worked with a partner throughout the 15-week semester. We then assigned to each student a child from the local community to whom they would provide direct services for 6 hr per week. A maximum of eight students, and therefore eight children, worked in the clinic each semester. Two Board Certified Behavior Analysts (2013) supervised all clinical services.

The clinic ran two back-to-back sessions that each lasted 90 min, Monday through Thursday. The children from each student pair were assigned to either the first or second session, which allowed each student to serve as the lead teacher for his or her own assigned student during one session, while providing support for his or her partner and the partner’s assigned student during the opposite session. This arrangement allowed for a 2:1 teacher-to-learner ratio. The lead teacher had primary responsibility for providing instruction, while the assistant teacher helped with gathering materials and taking data. After each session, both teachers worked together to make data-based instructional decisions based on each learner’s daily performance.

Materials

Teachers had a stack of laminated index cards and a grease pencil for use throughout each session. Additionally, teachers kept a clipboard with data sheets for making data-based decisions throughout each instructional session.

Response Definition

We assessed verbal behavior frequencies across two different learning channels: see/say and hear/say. Consistent with Skinner’s (1957) use of the word tact to describe verbal behavior under the control of extraverbal properties of the environment, participants demonstrated tact control when they would see the target and say its name. Additionally, consistent with Skinner’s (1957) use of the word echoic to describe verbal behavior under the control of intraverbal properties of the environment that share formal similarity, participants demonstrated echoic control when they would hear the target response and say it again. Finally, consistent with Vargas’s (1982) use of the word sequelic to describe verbal behavior under the control of intraverbal properties of the environment that do not share point-to-point correspondence, participants demonstrated sequelic control when they would hear a statement about the target response and say the name of the item.

Procedure

Prior to the children’s arrival on campus each semester, the authors met with teachers who enrolled in the practicum for 2 weeks to discuss clinical policy and procedure, and to provide a hands-on, interactive training based on the Registered Behavior Technician (RBT) Task List (Behavior Analyst Certification Board, 2013). The training lasted approximately 24 hr in duration and occurred at the same time of day when the teachers were scheduled to provide services to the children (i.e., 12 hr each week). Teachers spent the remaining 13 weeks of the semester providing RBI to clients from the local community.

As described by Mason and Andrews (2014), RBI is a treatment package that combines both natural environment training (NET) and frequency building to strengthen verbal behavior. RBI emphasizes the transfer of stimulus control across the verbal operants within the context of shaping novel responses. The overarching goal of RBI is that for every item of interest to the child, the child should be able to request it, label it, name it, and identify it by its primary feature(s). To achieve this goal, we used a combination of milieu and discrete-trial training, dividing each 90-min instructional session into nine 10-min intervals. Every interval consisted of 9 min of NET followed by a 1-min fluency probe.

Natural Environment Training

The first 9 min of each interval consisted of NET, in which the child engaged in some sort of play-based activity (e.g., swinging, stacking blocks, completing puzzles, drawing pictures, and playing with other age-appropriate toys). The items with which they engaged served as “referents” for further conditioning of language skills. That is, for each item under motivational control, we sought to teach corresponding tact, echoic, and sequelic control.

NET consisted of restricting access to the targeted stimulus and prompting the child to mand for access to the reinforcer. We used errorless language learning to condition each verbal response through an individualized system of most-to-least prompts. The initial prompt hierarchy for each child was derived from their pretest SCoRE assessment by ranking operants’ strengths from greatest to weakest. However, the prompting sequence was subject to change in accordance with the results from each subsequent fluency probe (described later).

For example, Fig. 1 shows the control ratio for Participant 39, for whom disproportionality across verbal operants was found to be statistically significant, X²⁽³⁾ = 23, p < .001. Visual analysis shows the rank order of strength, from greatest to weakest, as echoic, tact, mand, and sequelic. By converging all operants that were shown to have some degree of isolated strength, we maximized control over the participant’s verbal behavior. Participant 39 emitted at least one verbal response across all four conditions. Accordingly, we converged mand, echoic, tact, and sequelic control.

Fig. 1.

The stimulus control ratio for Participant 39. Results of the SCoRE assessment show that the participant’s verbal repertoire was primarily under the control of echoics and tacts, followed by mands and sequelics. At the time of the pretest, Participant 39 demonstrated a moderate repertoire of .53

This highest level of prompting was where we began errorless language learning for the participant. When the participant engaged with a toy during NET, we would (a) restrict access to the toy (to induce mand function), (b) present the toy as a visual stimulus (to induce tact function), and (c) provide a fill-in-the-blank frame (to induce sequelic function), while (d) modeling the target response (to induce echoic function). The intraverbal fill-in would then be immediately repeated to probe for the child’s response.

For instance, when Participant 39 played with a toy dinosaur, we restricted access to the item, showed it to him, and said, “Roar says the dinosaur. Roar says the ____.” When Participant 39 played with a toy train, we restricted access to the item, showed it to him, and said, “Choo goes the train. Choo goes the ____.” Similarly, when Participant 39 played on a slide, we interrupted the activity, showed it to him, and said, “You go down the slide. You go down the ____.”1

The teacher delivered the specified reinforcer upon each request, and we immediately moved to a lower level of prompting (see Table 1). For Participant 39, sequelic control provided the least amount of support; eliminating this type of prompting was our next step in the sequence. When Participant 39 played with a toy drum, we restricted access to the item, showed it to him, and said, “Drum.” When Participant 39 played with a toy car, we restricted access to the item, showed it to him, and said, “Repeat after me: ‘Car.’” Similarly, when Participant 39 played on a swing, we interrupted the activity, showed it to him, and said, “Say ‘swing.’”

Table 1.

Participant 39’s Individualized Prompt Hierarchy for Errorless Language Learning

Prompt Level	Example	Reinforcement
Converge mands, tacts, echoics, and sequelics (100%)	While the student is engaged with a ball, restrict access to the reinforcer (M) while keeping it in the student’s line of sight (T). Provide the target response (E) followed by an intraverbal frame (S): “You roll the ball. You roll the _____.”	Reinforce correct responding with access to the ball and verbal praise.
Converge mands, tacts, and echoics (96.2%)	While the student is engaged with a ball, restrict access to the reinforcer (M) while keeping it in the student’s line of sight (T). Provide the target response (E): “Say, ‘Ball.’”	Reinforce correct responding with access to the ball and verbal praise.
Converge mands, echoics, and sequelics (63.5%)	While the student is engaged with a ball, hide the reinforcer (M), and provide the target response (E) followed by intraverbal frame (S): “You roll the ball. You roll the _____.”	Reinforce correct responding with access to the ball and verbal praise.
Converge mands and echoics (59.6%)	While the student is engaged with a ball, hide the ball (M) and provide the target response (E): “Say ‘ball.’”	Reinforce correct responding with access to the ball and verbal praise.
Converge mands, tacts, and sequelics (55.8)	While the student is engaged with a ball, restrict access to the reinforcer (M) while keeping it in the student’s line of sight (T). Provide an intraverbal frame (S): “You roll the _____.”	Reinforce correct responding with access to the ball and verbal praise.
Converge mands and tacts (51.9%)	While the student is engaged with a ball, restrict access to the reinforcer (M) while keeping it in the student’s line of sight (T), and ask the student to request access: “What do you want?”	Reinforce correct responding with access to the ball and verbal praise.
Converge mands and sequelics (19.2%)	While the student is engaged with a ball, hide to the reinforcer (M) and provide an intraverbal frame (S): “You roll the _____.”	Reinforce correct responding with access to the ball and verbal praise.
Isolate mands (15.4%)	While the student is engaged with a ball, hide it and ask the student to request it (M): “What do you want?”	Reinforce correct responding with access to the ball.

Note. This sequence was generated from Participant 39’s pretest SCoRE assessment. As part of ongoing progress monitoring, the individual steps of the sequence were subject to change in accordance with the data from fluency probes. The percentages listed in parentheses show the converged proportion of control over the verbal repertoire.

This style of prompt fading continued according to the prompt hierarchy prescribed by the SCoRE, with the goal of conditioning independent mands during NET. Once the student demonstrated the ability to mand at a given prompt level, we immediately faded to the next less restrictive level of control. Individual targets were not required to be “mastered” across prompt levels; rather, we aimed to establish more generalized control by changing targets as soon as the student moved on to a new activity. In this way, RBI functioned as a continuous free-operant preference assessment to identify potential reinforcers for use in conditioning mand control (Sautter, LeBlanc, & Gillett, 2008).

Teachers allowed the children to freely select the activities throughout all NET activities. Children were not required to participate in any activity for a set duration of time, but they were required to clean up—with help from the teacher, as necessary—prior to engaging in a subsequent activity. Teachers provided opportunities to mand at a rate of 6 per minute or once every 10 s. A maximum of 81 min of each session consisted of NET.

Fluency Probes

The final minute of each 10-min interval consisted of a 1-min fluency probe. Throughout NET, teachers recorded the type of tact, echoic, and sequelic prompts they used to help condition independent mands. After probing for a mand—and while the students engaged with their newly acquired reinforcers—the teacher recorded the corresponding tact, echoic, and sequelic prompts specific to the reinforcing stimulus.

Developing tact and echoic prompts proved relatively simple, as they merely involved showing the student the reinforcing item or calling for the student to echo its name. Conversely, we found sequelic prompts somewhat more challenging to generate. Teachers were instructed to describe how the child engaged with the stimulus—not necessarily how the child should have engaged with the stimulus—as the basis for their fill-in-the-blank frames. Moreover, to prevent stimulus overselectivity, we also instructed teachers to develop novel S^Ds when children accessed the same reinforcer across consecutive sessions.

For example, one type of card prompted the teacher to show the item to the child and ask the child to label the item (e.g., Show the ball and say, “What do I have?”). Another card prompted the teacher to provide a verbal imitative stimulus to the child (e.g., Say, ‘Ball.’). The third type of card prompted the teacher to provide a fill-in-the-blank frame that corresponded to how the child engaged with the item (e.g., “You roll the ____.”). We repeated this process for each item with which the child engaged over the 9 min of NET. The index cards that amassed provided the set of stimuli for the subsequent 1-min fluency probe.2

At the end of every NET segment, we sat the children individually at one of the workstations with the teacher directly across from the child. Teachers shuffled their index cards to randomize the order of presentation and set a timer for 60 s. The teacher would then start the timer and quickly proceed through the deck of cards following the previously written instructions.

Throughout the timing, teachers stacked the index cards that evoked the targeted verbal response in one pile, and the cards that did not occasion the targeted verbal response in a separate pile. Cards that did not evoke any response within 5 s were also placed in the pile of incorrects. When the timer sounded at the end of the 1 min, the teacher would shout, “You did it!” and provide a hug or high five, before directing the student to play while the teacher quickly analyzed the data. Teachers conducted a maximum of nine fluency probes each day.

Data-based decision making

Fluency probes primarily served the function of assessing the efficacy of NET. The data from these probes also served as judgmental aids for making programmatic decisions. The count per minute of correct and incorrect tacts, echoics, and sequelics was counted and transferred to a data sheet that allowed for easy recognition of response patterns within and across fluency probes (see Fig. 2). These data tables allowed teachers to make in vivo instructional decisions about how to proceed with NET. The frequency of the highest verbal operants became an informal fluency aim for weaker operants. That is, data from the last fluency probe dictated how the teacher was to spend the next 9-min NET session.

Fig. 2.

An RBI data sheet showing the participant’s responses to various echoic, tact, and sequelic prompts across nine discontinuous 1-min fluency probes. Each timing assessed a variety of verbal operants, but the individual targets varied across each timing. As the 90-min session progressed, new targets were added to subsequent fluency probes

For instance, the first column of Fig. 2 shows that the participant’s verbal behavior is under relatively strong echoic control (n = 3), whereas the participant responded to fewer tacts (n = 2) and sequelics (n = 0). Logically, the teacher would spend the next 9 min transferring control with the use of echoic prompts in an errorless learning format. The randomization of targets during fluency probes ensured that teachers could not assess all verbal operants for each referent stimulus across timings. Importantly, RBI does not focus on the acquisition of individual targets; rather, the goal is to condition generalized stimulus control across each of these operants at proportionate levels of strength.

At the end of the session, teachers plotted tact, echoic, and sequelic data on a timings chart (Tpmin-4EC; see Fig. 3). They combined tact, echoic, and sequelic data for each fluency probe to assess the proportionality of verbal operants at a glance. Moreover, the level of responding for stronger operant(s) provided a visual target for weaker operant(s). Teachers then made data-based instructional decisions according to the learning pictures (Neely, 1982) that emerged both within and across sessions.

Fig. 3.

A timings chart showing a participant’s progress within and across RBI sessions. Learning pictures on the timings chart were used to make data-based decisions with respect to each participant’s instructional programming. Four different learning pictures are represented on here: Jaws (9/26, 9/27), Get Truckin’ (10/3), Dive (10/4, 10/6), and Takeoff (10/5)

Procedural fidelity was randomly assessed weekly using a 10-item checklist for 25% of all sessions for each teacher, according to the Behavior Analyst Certification Board’s intensive practicum supervision guidelines. Procedural fidelity was continuously assessed across the length of the study (M = 76.10%, Mdn = 80%). Supervisors provided remediation for all incorrect or omitted steps immediately after the session ended using an inquiry-based professional development model (Mason, Andrews, Rivera, & Davis, 2016).

Social Validity

At the conclusion of each participant’s enrollment in the project, parents completed a social validity questionnaire. These responses remained anonymous and could not be tied to any individual participant unless the parent provided identifying information. The questionnaire contained 17 items, including 8 open-ended questions (e.g., “Have your child’s treatment goals been met?” “What changes would you make to our program?” “Describe any changes in your child’s behavior outside of the center.”), 8 Likert-scale items with a range of 1 to 5, with 5 being the highest (e.g., “How would you rate the quality of our services?” “My child has benefitted as a result of this program.” “My child’s services are individualized to his or her strengths and needs.”), and 1 binary response (“Would you recommend us to others?”). Forty-three parent questionnaires were returned.

Results

Figure 4 shows pre/post data for the 49 participants. A paired-samples t test was used to determine whether there was a statistically significant mean difference between pre- and post-SCoRE assessments across 13 weeks of referent-based verbal behavior instruction. We detected one outlier of more than 1.5 times outside the interquartile range in the posttest SCoRE. Inspection of its values did not reveal it to be extreme (0.14), and therefore it was included in the analysis.

Fig. 4.

Box plots showing the average pretest (left) and posttest (right) SCoREs for 49 participants with language disorders across 13 weeks of referent-based verbal behavior instruction. The dark lines in each box represent the median SCoRE for both the pretest (Mdn = .53) and posttest (Mdn = .77)

A primary assumption of parametric testing is that sampled data follow a normal distribution. To assess whether the differences between the two paired observations met this assumption, we used the Shapiro–Wilk test of normality, which is appropriate for sample sizes less than 50. The resulting assumption of normality was not violated (p = .901), indicating that our data are normally distributed.

Participants demonstrated stronger verbal behavior on the posttest SCoRE (M = .700, SD = .221) when compared to the pretest SCoRE (M = .463, SD = .255), a statistically significant mean increase of .236, 95% CI [0.184, 0.289], t(48) = 9.0916, p < .001, d = 1.30. Surpassing the threshold of α = 0.05, the significant difference between pretest and posttest values stipulates that the mean pretest and posttest SCoRE values are not the same after 13 weeks of RBI.

Figure 5 shows pre/post data for each of the 49 participants, stratified according to pretest repertoire size. The average SCoRE of the 11 emergent speakers increased from .08 to .46, a difference of .38. The average SCoRE of the 11 participants who started with a practical speaking repertoire increased from .37 to .64, a difference of .27. The average SCoRE of the 23 participants who began with a moderate speaking repertoire increased from .62 to .80, a difference of .18. The average SCoRE of the 4 participants who began the study with strong speaking repertoires increased from .84 to .92, a difference of .08.

Fig. 5.

Box plots showing differences across pretest and posttest SCoREs for participants who began the study with emergent (>.20), practical (.20–.49), moderate (.50–.79), and strong (≥.80) verbal repertoires

Three outliers were identified in the pre-SCoRE data for students with moderate speaking repertoires, as assessed by inspection of a box plot for values greater than 1.5 standard deviations. We opted to include these outliers in the analysis after a comparison of results both with and without these outliers showed no substantial difference.

Responses to the open-ended questions on the social validity questionnaire were largely used for ongoing programmatic revisions over the length of the project. Across the eight Likert-scale items, parents reported a total of 312 fives (range 35–40), 31 fours (range 2–8), only a single three, and no ones or twos. When asked if they would recommend us to others, 100% of parents responded affirmatively.

Discussion

The current research sought to demonstrate the effects of RBI on conditioning proportionate stimulus control over the verbal behavior of early learners with language disorders. Across 5 years, 49 children with ASD participated in referent-based verbal behavior instruction, which focused on the transfer of stimulus control across echoic (hear/say), tact (see/say), and sequelic (hear/say) learning channels. The results of this study further support the use of a learning channel matrix (Haughton, 1980) to develop fluency aims. Binder (1996) explained that “the channel matrix enabled curriculum designers to plan for a variety of different skills in the same curriculum area by viewing all possible input/output combinations on a single summary form” (p. 181). Although previous research has supported the use of frequency aims from related skills within the same learning channel (Binder, 1996; McDade & Olander, 1990), our findings also recommend setting aims across learning channels for establishing fluent verbal behavior. However, we acknowledge that our findings are limited to elementary verbal behavior.

Aims for all verbal operants were individualized according to the strongest operant identified for each student. Although the strongest operant identified by each participant’s initial SCoRE assessment typically remained constant throughout the course of treatment, the actual performance criterion varied within and across sessions. The strongest verbal operant identified for each participant on the pretest SCoRE improved over time. Accordingly, the benchmark criterion for the other operants also increased. Data from the first fluency probe in Fig. 2 established an echoic benchmark of three correct responses. Later that session, the fifth fluency probe shifted to a tact benchmark of four. As such, omitting the standard convention of aim stars allowed for more dynamiclanguage development.

Additional research is required to better understand the relationship between SCoRE results and fluency aims. The context in which SCoRE assessments are administered differs greatly from fluency probes. Although the SCoRE assessment is useful for rank ordering the strength of verbal operants, it cannot be used to pinpoint specific fluency aims. However, knowledge of operant class ranks may be useful for predicting the results of timed fluency measures, and vice versa. Future research should seek to examine the correlation between SCoRE and curriculum-based assessments.

After 13 weeks of intervention, the average SCoRE for participants across all four repertoire sizes increased to the next level. That is, the average SCoRE of the 11 emergent speakers increased from .08 to .46, effectively moving them into the practical range. The average SCoRE of the 11 participants who started with a practical speaking repertoire increased from .37 to .64, effectively moving them into the moderate range. The average SCoRE of the 23 participants who began with a moderate speaking repertoire increased from .62 to .80, effectively moving them into the strong range. The average SCoRE of the 4 participants who began the study with strong speaking repertoires increased from .84 to .92, the threshold at which stimulus control discrepancies are no longer statistically significant (Cochran, 1950).

Each group’s gains varied in accordance with their pretest SCoRE. The lowest performing participants made the most significant gains, perhaps because they had the most room to grow according to the SCoRE measure. Moreover, the logarithmic influence of reinforcement on behavior has been well established (Kubina et al., 2002).

Five of these participants began the study with no functional speaking repertoire—a SCoRE of .00. Notably, bringing behavior into existence constitutes a remarkable change: Performance lives in the multiply world - grows and decays by multiplying and dividing," noted Lindsley (2000, August 16). Given the broad range of improvement demonstrated by these participants over time, additional research is necessary to address the genesis of verbal behavior. Although RBI emphasizes the transfer of stimulus control across the verbal operants, particular attention should be paid to the transfer of stimulus control from nonverbal to verbal operants and the extent to which proportionate levels of strength facilitate this transfer. For example, generalized motor imitation has been shown to induce language (Ross & Greer, 2003). However, the extent to which concomitant levels of matching to sample and manded stimulus selection facilitate this transfer has yet to be identified.

The gains made by practical, moderate, and strong speakers show decreasing returns in their postassessment SCoRE. This may be because shaping complex verbal repertoires requires greater finesse than shaping an emergent verbal repertoire. Future research should systematically examine these outcomes. Notably, the SCoRE assessment imposes a ceiling of 1.00, which likely constrained the growth of participants who started with a strong verbal repertoire.

Variation in participant performance is likely a result of the lack of consistency during the 9 min of instruction. The overall fidelity of implementation of these procedures suffered as a result of training novel therapists to work with participants each semester. Although the initial training covered declarative knowledge, we observed steep learning curves at the start of each semester as the teachers refined their instructional delivery and contingency management. Across semesters, two procedures consistently rated low on fidelity checks: (a) increasing prompts when participants responded incorrectly and (b) fading prompts when participants responded correctly. Prompting verbal behavior by converging multiple control, as well as fading prompts by diverging multiple control, on an individualized level is a complex technique that requires frequent practice to develop and maintain. Additional research should look into methods of facilitating the acquisition of these skills.

Consistent with prior research, results from the current study show that enhanced training on individual operants supports the transfer of control to other operants (Cihon, 2007; Egan & Barnes-Holmes, 2011; Finn et al., 2012; Petursdottir et al., 2005). Verbal operants were targeted for acceleration according to the individualized deficits demonstrated by each participant. Across the 49 participants who took part in this research, deficits were noted across all four verbal operants. It was beyond the scope of the current study to analyze the gains made by participants according to specific verbal behavior deficit(s). Future researchers should consider doing so, as such analyses may lead to more targeted treatment options.

The current research also did not compare the effects of single-operant training to those of multiple-operant training. Multiple-operant instruction through NET was used in conjunction with single-operant assessment via fluency probes to facilitate the transfer of stimulus control. Consistent with prior research on conditioning multiple verbal operants simultaneously, the results of the current research support the use of multiple-operant training to strengthen individual verbal operants (Arntzen & Almås, 2002; Carroll & Hesse, 1987; Kodak & Clements, 2009; Luccherino & Scali, 2013). Arguably, the repetition of fluency probes throughout each session may have presented a testing effect. Future research should systematically address the pairwise distinction between enhanced single-operant instruction and multiple control to better facilitate the transfer of control. Specifically, the extent to which the results of single-operant fluency probes can be used to established aims for RBI should be investigated, along with the extent to which celeration values correlate with the different repertoire sizes identified by the SCoRE.

The group-level outcomes presented here suggest that the benefits of RBI are generalizable to the population of children with ASD who show language deficits. However, our results also suggest the need for additional experimentation using single-subject research designs to rule out threats to validity and demonstrate a functional relationship between RBI and language fluency. Future researchers should examine the extent to which RBI can be used to build frequency to a performance criterion across each of the verbal operants.

Practitioners who teach verbal behavior to children with ASD are advised not to neglect the importance of the transfer of stimulus control. That is, although bringing a verbal response under the control of nonverbal stimuli is important—indeed, Skinner (1957) called it the most important operant—equally important are the speaker’s abilities to respond under relevant states of restricted access and verbal stimulus control. Therefore, we echo the call of Engelmann (2005), who noted that behavioral objectives do not readily translate into teaching programs. Behavioral objectives fail to account for other behavioral outcomes related to the original objective. In the case of teaching language, this means teaching verbal operants independently of one another. RBI corrects this stipulation by teaching the full range of contexts under which a given response is likely to access reinforcement.

The ultimate goal of this research was to demonstrate the effect of RBI on the development of the verbal behavior of children with ASD. As other researchers have observed, skills taught to an a priori performance aim do not reliably predict retention, endurance, stability, application, or adduction (Doughty et al., 2004; Fabrizio & Moors, 2003; Heinicke et al., 2010). Speaking of “fluency” in this way may be the result of a category error (Baum, 2017; Ryle, 1949). Specifically, maintenance, endurance, stability, application, and generativity are not a result of fluency; rather, the term fluency is a teleological descriptor that may be used to describe behavior that shows maintenance, endurance, stability, application, and generativity.

A similar mistake could be made in analyzing the results of the current study in saying that the use of RBI to teach verbal behavior to fluency results in homeostatic stimulus control over the verbal behavior of children with ASD. As it relates to the relative ease of transferring control across changing stimulus conditions, speaking fluency is better defined as a function of proportionate operant strength within the verbal repertoire of the speaker. Might balancing stimulus control similarly facilitate transfer across nonverbal operants? Let them extrapolate who will.

Compliance with Ethical Standards

Conflict of Interest

The authors have no conflicts of interest to declare.

Ethical Approval

The procedures performed involving human participants were in accordance with the ethical standards of the University of Texas at San Antonio’s Institutional Review Board (#2017-181) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.