The Effects of the Cool Versus Not Cool Procedure to Teach Social Game Play to Individuals Diagnosed with Autism Spectrum Disorder

Abstract

This study evaluated the utility of the cool versus not cool procedure for teaching three structured indoor games to eight children diagnosed with autism spectrum disorder (ASD). The study took place as part of a social skills group for individuals diagnosed with ASD, and this study was one component of that group. The cool versus not cool procedure consisted of the teacher demonstrating each game the cool (i.e., appropriate) and not cool (i.e., inappropriate) way and having the participants provide a rationale as to why the demonstration was either cool or not cool. This was followed by giving the participants the opportunity to role-play the game in front of the group. The teachers utilized unprompted performance probes with no programmed reinforcement to create opportunities for the participants to display the targeted behavior (s). A multiple baseline design across behaviors and replicated across participants was utilized. The results indicated that seven of the eight participants mastered each of the games taught.

Play can take many forms which include, but are not limited to, solitary play, parallel play, free play, video games, board games, and outdoor game play. Each form of play is essential for children, as play has been demonstrated to increase language development (Bodrova and Leong 1996), increase physical development (Frost et al. 2001), and improve a child’s overall development (Moore et al. 1992). Individuals diagnosed with autism spectrum disorder (ASD) have qualitative impairments in social behavior which may include limited play skills (American Psychiatric Association 2013). Given the importance play has on a child’s overall development, teaching individuals diagnosed with ASD play skills should be included in any comprehensive treatment package.

Today, there are several procedures that would be considered evidence based that are used to teach play skills to individuals diagnosed with ASD (Blum-Dimaya et al. 2010; Brodhead et al. 2014; Oppenheim-Leaf et al. 2012). Unfortunately, some professionals are endorsing and promoting non-evidence-based procedures to teach both play and social behavior to individuals diagnosed with ASD (e.g., Gray and Garand 1993; Winner 2008). For example, one commonly implemented intervention to teach social and play behavior for individuals diagnosed with ASD is social stories (e.g., Gray and Garand 1993). Today, many professionals utilize social stories even though the evidence to support their effectiveness is minimal and the majority of studies have not demonstrated a clear functional relationship (see Leaf et al. 2015b for a review). Another intervention that has been used to teach play and social skills to individuals diagnosed with ASD is Social Thinking® (Winner 2008). Social Thinking® has been promoted to teach social and play skills despite the dearth of empirical evidence demonstrating its effectiveness (see Leaf et al. 2016a). Additionally, proponents of Social Thinking® often make unsupported claims about interventions based upon the principles of applied behavior analysis (ABA) as an ineffective way to teach play and social skills to individuals diagnosed with ASD (Leaf et al. 2016a; Winner 2008). The abundance of procedures to choose from and the proliferation of non-evidence based procedures could result in professionals not implementing evidence-based procedures. As a result, children may not make the same amount of progress they could have made from an established evidence-based procedure.

Fortunately, to date, there are several evidence-based procedures to teach play that have been described in curriculum books for individuals diagnosed with ASD (e.g., Leaf and McEachin 1999) and evaluated empirically (e.g., Blum-Dimaya et al. 2010; Brodhead et al. 2014; Oppenheim-Leaf et al. 2012). One common procedure utilized to teach game play to individuals diagnosed with ASD is activity schedules (MacDuff et al. 1993). In 2014, Brodhead and colleagues investigated the effectiveness of an activity schedule to teach six participants diagnosed with ASD how to play hide-and-seek. The researchers created activity schedule binders for each of the participants and utilized a graduated guidance prompting procedure to teach the sequence of steps for the hide-and-seek game. The results demonstrated that the activity schedule binders were effective for teaching all six participants how to correctly play the hide-and-seek game.

A second procedure that has been utilized to teach game play is video modeling (Charlop and Milstein 1989). Paterson and Arco (2007) evaluated the use of video modeling to teach appropriate verbal and motor toy play to two children diagnosed with ASD. Each child was presented with the video model twice, which consisted of an adult engaging appropriately with the targeted toy. Immediately following the video model, the child was provided 3-min of access to the toy from the video. Anytime the child engaged with the toy appropriately during this time, the researcher provided verbal praise. The results of the study indicated that video modeling was effective at increasing appropriate verbal and motor toy play; however, the results did not generalize to unrelated toys. In a more recent study, Blum-Dimaya and colleagues (2010) evaluated the effectiveness of a video modeling procedure combined with activity schedules to teach two adolescents and one preadolescent how to play a popular video game. The author’s developed a task analysis of playing the video game consisting of 26 smaller steps. The activity schedule was used to teach the general set up of the game and video modeling was utilized to teach how to play the game. The results yielded positive effects, as all participants learned how to play the video game appropriately and the effects generalized to other tasks within the game.

Another common evidenced-based approach to teaching appropriate game play involves the student observing a teacher demonstrate the target behavior and the student role-playing within a structured setting. These procedures include behavioral skills training (BST; Miltenberger 2011), the teaching interaction procedure (TIP; Leaf et al. 2015a, b; Phillips et al. 1974), and the cool versus not cool procedure (e.g., Leaf et al. 2012a, b). Having a student watch the teacher demonstrate the appropriate response may be an important component in quality teaching as it creates an opportunity for the students to observe desired performance. Researchers have also found that teacher demonstration has led to improvements in social behavior (e.g., Leaf et al. 2012a, b). Having the student role-play the behavior is another important component of intervention that sets the occasion for the student to engage in the targeted behavior, allows the teacher to provide prompts to help ensure the student engages in the desired behavior, sets the occasion for reinforcement for engaging in the targeted behavior during the role-play, and allows for repeated practice; all of these are important components of learning a new skill.

The TIP (Leaf et al. 2015a, b; Phillips et al. 1974) uses role-playing and has been demonstrated to be effective in teaching play behavior. Oppenheim-Leaf et al. (2012) explored the effectiveness of a TIP within a group instructional format to teach two participants diagnosed with ASD how to play three structured rule-governed games (i.e., Go Fish, Uno, and Yahtzee Jr). The TIP consisted of identifying the skill, developing meaningful rationales for why the skill is important, breaking the skill down into its component parts, providing a teacher demonstration of the skill (appropriate and inappropriate), and the children role-playing the skills with feedback from the teacher. The result demonstrated the use of a TIP was effective at teaching each participant to play each of the games as displayed during both naturalistic and generalization probes.

Another procedure that often includes role-play as a component of teaching is the cool versus not cool procedure. The cool versus not cool procedure has been implemented clinically (e.g., Leaf et al. 2012a) and has been evaluated empirically (e.g., Leaf et al. 2012b) for individuals diagnosed with ASD. The cool versus not cool procedure is a discrimination program that consists of the teacher first demonstrating the target behavior both appropriately (i.e., cool) and inappropriately (i.e., not cool) while the learner observes. Next, the teacher asks the learner to identify whether the demonstration was cool or not cool and provide a verbal description of why. An optional component gives the participant an opportunity to role-play the behavior in front of the teacher. Throughout the procedure, feedback is provided based on the accuracy of response.

Leaf and colleagues (2012b) were the first to empirically evaluate the cool versus not cool procedure with individuals diagnosed with ASD. Within that study, the researchers evaluated the cool versus not cool procedure with and without the inclusion of the role-play component to teach a variety of social behavior. If the participants did not reach mastery criteria, the role-playing component was added following the teacher demonstration. The results showed that participants reached mastery criterion on 50 % of the targeted skills with demonstration only and an additional 37.5 % of targeted skills with the addition of the role-playing component. In a more recent study, Leaf and colleagues (2016b) evaluated the effectiveness of the cool versus not cool procedure with the role-play as a required component. In this study, the authors taught social interaction skills to three participants diagnosed with ASD. The results of the study showed that requiring the role-play component within the cool versus not cool procedure was an effective intervention to teaching all of the participants the targeted social interaction skills. Given the success of the cool versus not cool procedure at teaching various social skills, it may prove fruitful to examine its effectiveness in teaching game play, which are often social in nature.

The cool versus not cool procedure may have some distinct advantages for teachers when teaching game play to individuals diagnosed with ASD. First, the cool versus not cool procedure requires minimal to no material preparation, which is unlike video modeling or activity schedules. Minimal to no prior material preparation increases the time a teacher has to provide learning opportunities rather than organizing and gathering materials. Second, the cool versus not cool procedure creates the opportunity for participants to observe an appropriate and inappropriate demonstration of the desired behavior. This provides the child with a discrimination of not only the preferred way of responding but also the non-preferred way of responding. Finally, the cool versus not cool procedure may be an easier procedure for therapists to learn and implement than other commonly implemented procedures (e.g., behavioral skills training or the teaching interaction procedure); thus, it can potentially be implemented by a wider range of staff.

Given the potential advantages of the cool versus not cool procedure, the preliminary evidence demonstrating its effectiveness to teach social behavior, and the need to identify more procedures to teach game play, empirical evaluation of the cool versus not cool procedure to teach game play behaviors is warranted. Therefore, the purpose of this study is to evaluate the effects of the cool versus not cool procedure with a required role-play component, implemented in a group instructional format, on teaching structured indoor games to eight individuals diagnosed with ASD.

Methods

Participants

Eight participants independently diagnosed with ASD participated in the study. All children had full scale IQ scores above 80, had well-developed conversational repertoires, and displayed minimal aberrant behavior. However, all participants demonstrated significant delays in social behaviors (i.e., less than 8th percentile) as indicated by the Social Skills Improvement Scale (SSIS). All participants were enrolled in a behaviorally based social skills group for individuals diagnosed with ASD. None of the participants had a previous history with the procedures implemented within this study or participating in behaviorally based social skills groups. Table 1 provides demographic information for each participant.

Table 1.

Participant demographics

Name	Age	Sex	WPPSI-III full scale IQ score	Vineland Adaptive Behavior Scale Adaptive Score	Social Skills Improvement System  Standard Score and Percentile Rank	SRS T Score
Sal	5	Male	92	97	58	66
					<1 %
Alex	5	Male	101	85	58	64
					<1 %
Joseph	3	Male	91	77	54	70
					<1 %
Ryan	3	Male	110	77	45	71
					1 %
Gabe	5	Male	100	75	63	67
					1 %
Greg	4	Male	119	77	69	57
					3 %
Carol	5	Female	91	75	76	72
					7 %
David	4	Male	107	111	57	74
					<1 %

Setting

This study was one component of a larger behaviorally based social skills group for young individuals diagnosed with ASD. The social skills group, and this specific research study, took place in a large clinic room as part of a private agency that provides comprehensive behavioral intervention for individuals diagnosed with ASD. As part of the social skills group, the teachers implemented a “behavioral thermometer” (BT; differential reinforcement system with response cost) which was conducted both within the context of this study and outside of the study. The BT was a visual system with various levels ranging from “superstar student” to “miss a fun activity.” Each participant had their own clothespin that would be moved up the thermometer for engaging in the desired behavior and down the thermometer for engaging in any undesired behavior. The BT was used within the context of this study but only occurred during the teaching component (described below) and not during any of the probes (described below).

Staff

This study, and the entire social skills group, was overseen by a doctoral level behavior analyst (referred to as researcher-second author) who had over 15 years of experience working with individuals diagnosed with ASD. The researcher was in charge of recording behaviors during probes and intervention. The probes and intervention were implemented by three professionals, each of whom had over 5 years of experience working with individuals diagnosed with ASD and implementing applied behavior analytic interventions (referred to as teacher or support teachers, the first, third, and fourth authors). During each session, one of the teachers would take the role of lead teacher and the other two teachers acted as support teachers and the roles were rotated each session.

Skills Taught and Response Definitions

Three structured games were selected to be taught during the intervention (i.e., sleeping game, fruit salad, and mouse trap). The games were selected based on a hypothesis that they would result in peer reinforcement, set the occasion to target general compliance, and provided the opportunity to target learning to learn skills embedded within each of the games. Each game was task analyzed into their component steps.

Sleeping Game

The first game that the teachers targeted was the sleeping game. To begin the game, the lead teacher had the participants sit on the floor in a semicircle facing the lead teacher. Next, the lead teacher provided an instruction to all of the participants on how to play the game. The instruction that was provided was the following statement:

“We are going to play the sleeping game. I want you to lie on the ground and pretend to sleep while [name of second teacher] hides a stuffed animal of a dog. I am going to make different animal sounds and you should stay asleep until you hear me bark like a dog. When I bark like a dog I want you to wake up and search the room for the dog.”

After the lead teacher provided this instruction, she or he told the participants to go to sleep and started to make various animal sounds (e.g., “moo,” “cluck-cluck,” “meow,” etc.). At the same time, the support teacher hid a stuffed animal of a dog somewhere in the room. Once the lead teacher barked like a dog (i.e., “woof-woof”) the participants were to “wake up” and search for the dog. Once any participant found the dog and brought it back to the lead teacher, the lead teacher told the participants to go back to sleep and the next trial began. Subsequent trials were implemented identical to the first trial with the exclusion of the instruction statement. This continued until there were three opportunities for the participants to find the stuffed dog.

For a participant to play the sleeping game appropriately he or she had to demonstrate the following steps on each of the three opportunities: (a) lay face down until the teacher barks like a dog, (b) have his or her eyes closed until the teacher barks like a dog, (c) sit up and open his or her eyes once the teacher barks like a dog, (d) walk around the room looking for the hidden stuffed animal until one member of the group finds the animal, and (e) only the participant who found the dog makes a statement that he or she found the stuffed animal (e.g., “I found it!”). A participant had to display all of these steps correctly on all three opportunities for the researcher to score that participant as playing the game appropriately.

Fruit Salad

The second game that was targeted was fruit salad. Fruit salad consisted of a total of four trials. To begin the game, the lead teacher had each of the participants sit on a chair. The support teachers arranged the chairs in two rows of four with the participants facing each other. The rows were approximately 10 ft apart. The lead teacher explained how to play the game to the participants by providing the following statement:

“We are going to play fruit salad. I am going to tell each of you the name of a fruit. When I call your fruit I want you to run into the middle and give a high-five to your friend. When I say fruit salad I want everyone to run into the middle and high five each other.”

The lead teacher then assigned a fruit to each participant by touching the participant on the head and saying, for example, “You are an apple.” Fruit assignments were counterbalanced and equal across the two rows (e.g., each row had one apple, one orange, etc.); therefore, four different fruits were used during the game. The lead teacher then called out one fruit at a time (e.g., “Apple”) and provided approximately 5 s for the participants to respond. The lead teacher continued until all fruits were called out, thus completing one opportunity for the participants to respond. This continued until the participants had three opportunities to respond to their fruit. After each participant had three opportunities the lead teacher would call out “fruit salad” and provided 5 s for the participants to respond.

For a participant to play fruit salad appropriately, he or she had to demonstrate the following steps during every opportunity to respond: (a) stand up from his or her chair within 5 s of the lead teacher stating his or her fruit or “fruit salad;” (b) remain seated when other participants’ fruits are called; (c) run to the middle of the two rows of chairs and provide a high five to the other participant whose fruit was also called; and (d) sit back down in his or her chair within 5 s of giving the other participant a high five. A participant had to display each of these steps on all four opportunities (i.e., three times her or his fruit was called and one time fruit salad was called) for the researcher to score the participant as playing the game appropriately.

Mouse Trap

The final game that was targeted was mouse trap. Mouse trap started with both of the support teachers bringing out a large toy parachute. The lead teacher explained the game by providing the following statement:

“We are going to play mouse trap. I want you to each hold a handle of the parachute and keep holding the handle until I call your name. When your name is called I want you to run under the parachute to the other side and hold the handle. You should not run to the other side until your name is called.”

The lead teacher then called up the participants one at a time and had them hold a handle of the parachute with at least one hand. Once all participants were holding a handle of the parachute, the lead teacher would begin by calling a participant’s name. The lead teacher provided the participant 5 s to run under the parachute to the other side. The lead teacher continued to call out names until each participant had been called. While the names were being called, all participants and the two support teachers moved the parachute up and down in an attempt to “trap” the running participant. This continued until all participant names were called one time; thus, each participant had only one opportunity within the game.

For a participant to play the mouse trap game appropriately, they had to demonstrate the following steps: (a) hold onto the parachute with at least one hand, (b) refrain from going under the parachute until his or her name was called, (c) go under the parachute within 5 s of his or her name being called, (d) leave the parachute within 5 s of going under the parachute, (e) hold onto a handle once leaving the parachute, and (f) not go back under the parachute for the remainder of the game. A participant had to display each of these steps for the researcher to score the participant as playing the game appropriately.

Dependent Variables

The main dependent variable was appropriate game play (defined above for each game). Mastery criterion was determined by performance probes (described below). Mastery was defined as the participant playing the game appropriately (described above) across three consecutive performance probes. We also evaluated the number of sessions required for participants to reach this mastery criterion on each of the three games.

Experimental Design

The researchers utilized a multiple baseline design across behaviors (games) replicated across the eight participants. The design consisted of three conditions: baseline, intervention, and maintenance. Within this design, intervention was not implemented on the second or third game until: (a) there was stable responding in baseline levels for the entire group and (b) there was an increase in the number of participants engaging in appropriate game play for the previous game. Thus, there were periods of times where multiple games were being taught simultaneously. When a skill was in baseline or in maintenance, the researchers only implemented performance probes (described below). When a skill was in intervention, the researchers implemented both a performance probe (described below) and the intervention (described below).

Performance Probes

Performance probes were opportunities for the researchers to evaluate the participants’ responses free from direct instruction, prompting, and programmed consequences. During all performance probes, the teacher provided the instruction of how to play the game (described above) in an effort to determine if the participants would respond to the instruction only.

Performance probes for the sleeping game consisted of the lead teacher providing all participants the instruction of how to play the game (described above) followed by the lead teacher instructing the participants to play the game. Performance probes for the sleeping game consisted of three opportunities for the participants to find the stuffed dog. During performance probes, the researchers scored if each participant played the game appropriately as described above. Performance probes for fruit salad consisted of the lead teacher providing all participants the instruction of how to play the game (described above) followed by the lead teacher instructing the participants to play the game. Performance probes for fruit salad consisted of three opportunities for the participants to respond to his or her fruit and one opportunity to respond to fruit salad. During performance probes the researchers scored if each participant played the game appropriately as described above. Performance probes for mouse trap consisted of the lead teacher providing all participants the instruction of how to play the game (described above) followed by the lead teacher instructing the participants to play the game. Performance probes for mouse trap consisted of one opportunity for the participants to run under the parachute. During performance probes, the researchers scored if each participant played the game appropriately as described above.

General Procedure

At the beginning of each social skills group the lead teachers and support teachers implemented performance probes (described above) for each of the three games; the order of probes varied each session. The probes were followed by a period of time in which the participants engaged in other activities not associated with this study (e.g., outdoor free play or learning joint attention). Then, the lead teacher implemented the intervention for the game (s) currently in the intervention condition. If there were multiple games in intervention, the teachers targeted one game, provided a brief 5 min break, and then targeted the next game.

Cool Versus Not Cool Procedure

To teach each of the three games, the social skills group teachers implemented the cool versus not cool procedure with a required role-play component (Leaf et al. 2012a, b, 2016b). This procedure consisted of teacher demonstration of the appropriate and inappropriate way to play the game followed by the participants role-playing the appropriate game. The lead teacher utilized in-the-moment assessment (Leaf and McEachin 1999; Leaf et al. 2016c, d) to determine how many teacher demonstrations were necessary, which resulted in a varied number of teacher demonstrations per session. The number of teacher demonstrations ranged from two to five, with an average of 3.7, 3.6, and 3.2 for the sleeping game, fruit salad, and mouse trap, respectively (see Table 2 for total teacher demonstrations per session). The lead teacher determined the number of teacher demonstrations to implement based upon responding during the performance probes, attending during teaching, and responding during teaching. Within each teaching session, there were at least one cool demonstration and one not cool demonstration.

Table 2.

Number of demonstrations per teaching session

Session	Sleeping game teacher demonstrations	Fruit salad teacher demonstrations	Mouse trap teacher demonstrations
Session 1	5	5	4
Session 2	5	5	3
Session 3	5	5	4
Session 4	3	5	2
Session 5	3	4	–
Session 6	4	4	–
Session 7	2	4	–
Session 8	4	3	–
Session 9	4	2	–
Session 10	4	2	–
Session 11	2	4	–
Session 12	–	2	–
Session 13	–	2	–
Average number of teacher demonstrations	3.7 per session	3.61 per session	3.25 per session

Table 3 provides an overview of how the cool versus not cool procedure was implemented within the context of this study. The demonstrations and the role-plays were set up similarly to the performance probes (see above). During the demonstration trials, the two support teachers pretended to be children in the group and the lead teacher led the game as s/he typically would. During cool demonstrations, the two support teachers displayed all of the steps (described above) of the game correctly. During not cool demonstrations, the two support teachers either omitted one of the steps (e.g., not looking for the stuffed animal during the sleeping game) or displayed a step incorrectly (e.g., standing up when an apple was called when the support teacher was assigned an orange during fruit salad). Incorrect demonstrations were selected based upon participant (s) performance during the performance probes. That is, the support teachers simulated mistakes made by participants during the performance probe earlier that session.

Table 3.

Steps of the cool versus not cool procedure

Steps	What occurred
Students sitting	The teacher ensured that all participants were sitting in a semi circle on the floor
Students facing the teacher	The teacher stated the names of the participants or group name to have them orientate towards the teacher.
Explanation of what was being taught	Teacher stated what will be worked on. E.g., “ Today we are going to practice playing fruit salad”.
Explanation of how teaching will occur	Teacher stated that the teachers were going to practice first and then it would be the participants turn to practice.
Starting the demonstration trial	Teacher stated “Now you are going to watch the teachers. Ready set action.”
The demonstration trial	Two teachers demonstrated the targeted game play either the cool or not cool way in front of the entire group.
Ending the demonstration trial	The lead teacher said “Cut” to end the demonstration trial.
Discriminating cool versus not cool	The teacher asked the entire group whether the demonstration was cool or not cool. The teacher had all students give a thumbs up for a cool demonstration and a thumbs down for a not cool demonstration.
Reinforcement	The teacher provided social praise and/or moved the participant up the level system for correct responding.
Punishment	The teacher provided corrective feedback and/or moved the participant down the level system for incorrect responding.
Explanation of why cool versus not cool	The teacher selected at least three students to state why the demonstration was cool or not cool
Reinforcement	The teacher provided social praise and/or moved the participant up the level system for correct responding.
Punishment	The teacher provided corrective feedback and/or moved the participant down the level system for incorrect responding.
Role-plays	All participants got to play the game similar to the naturalistic probes. The teachers provided reinforcement, priming, and prompting during role-plays, as needed.

Role-plays were identical to performance probes with two exceptions. First, the teachers provided feedback for correct and incorrect responding following a role-play. Reinforcement included verbal feedback and the teachers moving the participant’s clothespin up the BT. Corrective feedback included verbal feedback and the teacher moving the participant’s clothespin down the BT. Second, the teachers provided verbal prompts utilizing a flexible prompt fading procedure (Soluaga et al. 2008) during role-play probes if the participant responded incorrectly during previous role-plays or if the participant was not attending during the trial. During the majority of sessions, all participants role-played together in a large group; however, there were some sessions in which participants role-played in smaller groups.

Reinforcement

The social skills group teachers used the BT as part of the cool versus not cool procedure. The thermometer was never used during performance probes. The teachers moved the participants’ clothespins up the BT for attending during instruction, answering questions correctly, and role-playing correctly. The teachers moved the participants’ clothespins down the BT for not attending, engaging in aberrant behavior, answering questions incorrectly, and role-playing incorrectly.

Interobserver Agreement and Treatment Fidelity

One of the social skill teachers scored each performance probe for each of the three games across all sessions. A secondary observer collected interobserver agreement (IOA) data during 41.2 % of performance probes for the sleeping game, 33 % of the performance probes for fruit salad, and 44.4 % of performance probes for mouse trap. To collect IOA data, we evaluated each of the steps of the game (described above), for each of the opportunities, across all participants. For the sleeping game and fruit salad, IOA was calculated by dividing the number of trials in which there was an agreement that the participant displayed the game correctly or incorrectly by the total number of trials (i.e., agreements plus disagreements). For mouse trap, IOA was calculated by totaling the number of agreements (i.e., steps where the two observers agreed that the behavior occurred or did not occur) divided by the total number of steps. For the sleeping game, IOA was 98.2 % (range, 91.6 to 100 % per session); for fruit salad IOA was 95.1 % (range, 87.5 to 100 % per session); and for mouse trap IOA was 97.9 % (range, 88.9 to 100 % per session).

To assess treatment fidelity, an independent observer recorded planned teacher behaviors during 32.5 % of all teaching sessions. Planned teacher behaviors consisted of the following: (a) labeling the skill to be taught, (b) providing a cue to start each teacher demonstration, (c) implementing at least one cool demonstration, (d) implementing at least one not cool demonstration, (e) providing a cue to end each teacher demonstration, (f) having the participants state whether the demonstration was either cool or not cool, (g) providing the correct feedback (reinforcing or corrective) based upon the discrimination, (h) having at least three participants state why the demonstration was either cool or not cool, (i) having all of the participants role-play the game, and (j) providing correct feedback (reinforcing or corrective) based upon their performance during the role-play. To determine treatment fidelity, the researchers calculated the number of sessions in which the teachers demonstrated all of the behaviors correctly over the total number of sessions; the teachers implemented all planned behaviors during 100 % of sessions.

Results

Figure 1 displays the cumulative record of when participants reached mastery criterion for each of the three games. Figure 2 displays the percentage of participants that played the game correctly during performance probes (described above) during each session across all three conditions. To calculate this number, we divided the total number of participants who displayed the game correctly by the total number of participants who played the game correctly and incorrectly. This data was used to determine when teaching of a new game could begin.

Fig. 1.

Cumulative graph of participants reaching mastery criterion

Fig. 2.

Percentage of participants playing games correctly

Figures 3, 4, 5, and 6 display the skill acquisition, as measured through performance probes, for each participant individually. Each figure depicts the data for two participants; one participant’s data is in the left column and another participant’s data is in the right column. Each skill is divided into four conditions: (a) baseline, (b) intervention, (c) post, and (d) maintenance. Post represents data collected after a participant had already reached mastery criterion but continued to participate in teaching because other members of the group had not yet reached the mastery criterion.

Fig. 3.

Sal’s and Alex’s skill acquisition data

Fig. 6.

Carol’s and David’s skill acquisition data

For the sleeping game, none of the participants engaged in the steps of the game correctly during baseline performance probes. Seven of the eight participants reached the mastery criterion during performance probes for the sleeping game (see Fig. 1). Additionally, an increasing trend of the percentage of participants displaying the steps of the game correctly can be observed during the intervention condition (see Fig. 2). Finally, in the maintenance condition, all eight participants (including one who did not reach mastery criterion) responded correctly on all performance probes. Across the seven participants who reached the mastery criterion, an average of 7.57 teaching sessions (range, 5 to 11 sessions) were required to reach mastery criterion and an average of 7.14 sessions (range, 5 to 11 sessions) were required for a participant to reach mastery criterion when absences are factored out. The one participant who did not reach mastery criterion, Ryan (see Fig. 4), did show improvement in performance from baseline, but did not reach the mastery criterion.

Fig. 4.

Joseph’s and Ryan’s skill acquisition data

For fruit salad, none of the participants engaged in the steps of the game correctly during baseline performance probes. Seven of the eight participants reached mastery criterion during performance probes for fruit salad (see Fig. 1). Considerable variability occurred in the intervention condition, but a clear level change in the percentage of participants engaging in the steps of the game correctly can be observed during the intervention condition (see Fig. 2). Finally, during the maintenance condition, all eight participants (including one who did not reach mastery criterion) engaged in high rates of correct responding. Across the seven participants who reached mastery criterion, an average of 6.42 total teaching sessions (range, 3 to 13 sessions) were required to reach mastery criterion and an average of 5.42 sessions (range, 3 to 11 sessions) were required for a participant to reach mastery criterion when absences are factored out. It should be noted that while one participant did not reach mastery criterion, Ryan (see Fig. 4), there was an increase in overall performance. Finally, after participants reached mastery criterion, there were variable responses in the post-intervention condition prior to the maintenance condition (see Figs. 3, 4, 5, and 6).

Fig. 5.

Gabe’s and Greg’s skill acquisition data

For mouse trap, four of the participants (i.e., Alex, Joseph, Ryan, and Gabe) did not engage in the steps of the game correctly during the baseline condition while three of the participants (i.e., Sal, Greg, and David) engaged in the steps of the game correctly in at least one of the performance probes during baseline. Carol reached mastery criterion for mouse trap in the baseline condition and learned the game from instruction only. All eight participants reached mastery criterion for the game of mouse trap (see Fig. 1) and all participants displayed a high percentage of correct responding during the intervention and maintenance conditions (see Fig. 2). Across the seven participants who reached mastery criterion during intervention, it took an average of 3.42 total teaching sessions (range, 3 to 4 sessions) to reach mastery criterion and an average of 3.28 sessions (range, 3 to 4 sessions) for a participant to reach mastery criterion when absences are factored out.

Discussion

The purpose of this study was to evaluate the effectiveness of the cool versus not cool procedure implemented in a group instructional format to teach three games to eight individuals diagnosed with ASD. Seven participants reached the mastery criterion on all three games and one participant reached mastery criterion on one of the three games. The participant who did not reach mastery criterion did display higher rates of correct responding relative to baseline and did play the game accurately during the assessment of maintenance. Overall, the results of the study indicated that the cool versus not cool procedure was effective in teaching eight individuals diagnosed with ASD three different games.

This study has several implications for clinicians working with individuals diagnosed with ASD. First, this study provided further empirical evidence for the overall effectiveness of the cool versus not cool procedure. To date, there have only been two published empirical studies (Leaf et al. 2012b, 2016b) demonstrating the effectiveness of the procedure. Therefore, the cool versus not cool procedure does not yet meet the standards for best practice (Horner et al. 2005). Thus, additional research on the procedure increases our confidence in its effectiveness as a procedure for teaching individuals diagnosed with ASD. Establishing the effectiveness of this procedure provides practitioners with additional options when designing interventions for individuals diagnosed with ASD. Second, this is the first study that has evaluated the utility of the cool versus not cool procedure to teach game play to individuals diagnosed with ASD, which provides clinicians with an alternative approach to teach games.

Third, in this study, the baseline condition showed that the majority of participants were unable to play the games correctly after hearing the rules to the game. These results indicate that children diagnosed with ASD with rather high IQ scores (i.e., above 80), well-developed conversational repertoires, and minimal aberrant behavior may require systematic instruction to learn skills that their neuro-typical peers may learn more naturally. On the final game, mouse trap, one participant was able to learn from instructions alone and three participants were able to display the game correctly on some probes with instructions alone. It is possible that implementation of the cool versus not cool procedure resulted in generalized learning.

This study also has implications for future research. First, each of the games consisted of multiple embedded instructions. Within the sleeping game, there is potential to target such skills as: (a) waiting, (b) conditional instructions, (c) attending, (d) auditory attending, (e) auditory discrimination, (f) environmental awareness, (g) persistence, (h) avoiding peer pressure, (i) labeling, and (j) social enjoyment. The fruit salad game provides the opportunity to target such skills as: (a) social engagement, (b) auditory attending, (c) auditory discrimination, (d) social excitement, (e) waiting, (f) receptive instructions, and (g) conditional instructions. For the mouse trap game, there could be multiple embedded instructions, which include the following: (a) responding to one’s name, (b) waiting, (c) auditory attending, (d) auditory discrimination, (e) social engagement, and (f) following instructions. Although there is the potential for multiple embedded instructions within each of the targeted games, they were not evaluated within the context of this study. Future researchers may wish to evaluate if improvements in these skills are observed after children learn the games.

In this study, one participant, Ryan, was unable to reach the mastery criterion for two of the games. Anecdotally, this participant displayed higher levels of inattention, gazing, and stereotypic behavior. This could contribute to lower rates of correct responding. Future investigators could evaluate what participant characteristics increase or decrease the likelihood that the cool versus not cool procedure would be an effective intervention. This type of evaluation may lead to identification of what prerequisite behaviors are necessary for a student to learn from the cool versus not cool procedure.

Despite the positive findings of this study and potential benefits for clinicians and researchers, some limitations are worth noting. First, although one of the participants displayed generalized learning on the final game, it does demonstrate participant learning prior to intervention. Therefore, an argument can be made that the design failed to demonstrate functional control of the intervention. However, this argument can only be made for one of the eight participants. Therefore, it is unlikely that the intervention was not responsible for the changes in participant response despite the generalized learning displayed by one participant. Second, this study was part of a larger social skills group, so it is not known what affect the social skills group had on overall learning. It is possible that without the entire group intervention (e.g., reinforcement throughout the day, teaching other social behaviors, instructional control) children may not have mastered the targeted games with the teaching procedure alone. Third, generalization was only assessed during performance probes without prompting and programmed reinforcement. Future researchers should include more socially valid generalization measures (e.g., if the learned skills are displayed in other settings, with other kids, with other games, etc.) Demonstrating that the skills generalize to the participants’ school placement would further demonstrate that ABA-based procedures could be effective in changing play behavior across settings. More importantly, future researchers should evaluate if learning these games result in an improvement in the embedded behaviors within the game; as this would demonstrate that teaching play can improve socially significant behaviors.

Another limitation is that the teachers used a varying number of teacher demonstrations within this intervention which allowed for the use of clinical judgment and in-the-moment assessment (Leaf et al. 2016c). Although some authors have stated these are characteristics of quality, progressive teaching (Leaf et al. 2016d), it does decrease the technological soundness of the intervention (Baer et al. 1968, 1987) and, therefore, makes the results more difficult to replicate. Future researchers could examine the possibility of creating protocols to teach in-the-moment assessment and the development of tools to ensure that professionals are implementing these procedures with a high degree of fidelity.

Future researchers may wish to evaluate the procedure in different settings and with a variety of different participants. In this study participants all displayed minimal stereotypic behaviors that could interfere with learning. Each of the participants had high levels of receptive and expressive language which may help increase the likelihood of students learning the behavior. Evaluating the procedure with participants with different characteristics may help identify the conditions under which the cool versus not cool procedure is effective for teaching individuals diagnosed with ASD. Despite these limitations, the present study demonstrated the effectiveness of the cool versus not cool procedure in teaching game play for individuals diagnosed with ASD.

Compliance with Ethical Standards

This study was partially funded by the Organization for Autism Research. All authors received funding to conduct the study. The seventh, eighth, and ninth author have commercial products (i.e., curriculum books and DVDs) for the procedures implemented and curriculum utilized. All procedures performed in studies involving human participants were in accordance with ethical standards of the institutional research committee and with 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Informed consent was obtained from the parents of all individual participants included in the study. Formal assent was also obtained from all individual participants.