Evaluating Remote Behavioral Skills Training of Online Gaming Safety Skills

Abstract

This study assessed the use of behavioral skills training for teaching online safety skills to two children, 7 and 9 years old. In situ assessments took place while playing the popular online game, Among Us, and consisted of confederates presenting lures to the child participants. Before training, the participants scored 1s and 0s, indicating a lack of online gaming safety skills. Both participants scored at mastery criterion (i.e., three scores of 3 in a row) following training. During the first 2-week follow-up, one participant scored a 1 because he did not leave the game following the presentation of a lure; his mother immediately implemented in situ training (IST). He scored at mastery criterion during the next follow-up assessment, indicating the effectiveness of the caregiver-implemented IST. The second participant scored a 3 during his 2-week follow-up.

Children and adolescents spend a large portion of their time online. An annual report by the U.S. National Center for Education Statistics (2020) found that 94% of 3- to 18-year-olds had access to the internet while home in 2018. Many of these children and adolescents interact with others over the web daily. The results of a national survey of teens 13 to 17 years old, conducted by the Pew Research Center, found that 57% of the surveyed teens had made a new friend online (Lenhart et al., 2015). Although social media (e.g., Facebook, Instagram) is the most common place for youth to meet new people, 36% of the 1,060 participants in the survey had met a new friend on an online video game (Lenhart et al., 2015). While online, players can seamlessly interact with others from around the world. Although many interactions with strangers online can be benign, others can lead to dangerous encounters. Madigan et al. (2018) conducted a meta-analysis of studies reviewing online sexual exposure (37,649 participants total) and solicitation (18,272 participants total) to minors. The researchers found that one in five youth encounter unwanted sexually explicit material while online, and one in nine youth experience sexual solicitation from strangers online.

Cernikova et al. (2018) interviewed 9- to 16-year-olds in the Czech Republic to gather information on their experiences with strangers online. Some negative experiences included strangers asking personal questions or soliciting sexual behavior. Although a small percentage of youth who meet strangers online actually contact them in real life, the potential is still a cause for concern (Livingstone & Smith, 2014). Zhang-Kennedy et al. (2016) interviewed 14 parent–child dyads to assess their experiences and perceptions about internet usage; all but one parent viewed “stranger danger,” such as sharing private information with unidentified strangers, as a major concern when their children were online. Despite parents’ fears, research shows that children do not always know what safety threats exist online or have the knowledge or skills to respond to online safety threats (Macaulay et al., 2020; Zhang-Kennedy et al., 2016).

Despite the prevalence of youth accessing the internet and interacting with strangers, there is limited research assessing the safety skills of youth when online or teaching online safety skills. Boulton et al. (2016) examined the effectiveness of older students utilizing the information-based Cross-Age Teaching Zone (CATZ) intervention to teach younger students’ knowledge of online risks and safety. The older students (i.e., the tutors) developed 30-min lessons on online risks and how to safely avoid them and presented them to younger students (i.e., the tutees). The results of the study showed that CATZ tutors and tutees scored higher when answering open-ended questions related to online risks and safety compared to participants who did not receive CATZ training. Although the participants scored high on these knowledge measures, it is unclear if participants would have demonstrated the safety skills if presented with a lure in a real online game because they were not given the opportunity to practice the skills.

Research on teaching abduction prevention skills (and other safety skills) to children has demonstrated that active learning approaches such as behavioral skills training (BST) and in situ training (IST) are most effective. BST involves instructions, modeling, rehearsal, and feedback to teach skills. In teaching abduction prevention skills, the researcher describes the different types of abduction lures and describes the skills the child needs to respond safely to these lures. The researcher then models the skills in role plays that simulate abduction situations so the child sees multiple exemplars of the safety skills. Next, the researcher has the child rehearse the skills in role plays in which abduction lures are simulated and the child engages in the three skills described earlier. In the context of these rehearsals, the researcher provides praise for correct performance and corrective feedback involving further instruction for any aspect of performance that was incorrect. Over multiple training trials, the child engages in the safety skills until correct performance occurs multiple times. Researchers have shown that BST can be successful to teach abduction prevention skills (Carroll-Rowan & Miltenberger, 1994; Gunby et al., 2010; Holcombe et al., 1995; Johnson et al., 2006; Miltenberger & Thiesse-Duffy, 1988; Poche et al., 1981; Poche et al., 1988; Sanchez & Miltenberger, 2015) and other safety skills such as firearm safety skills (Gatheridge et al. 2004; Gross et al., 2007; Himle et al., 2004a, b), and poison safety skills (Dancho et al., 2008; Morosohk & Miltenberger, 2022; Petit-Frere & Miltenberger, 2021; Rossi et al., 2017).

Although BST is often an effective intervention for teaching safety skills, some research has shown that BST does not consistently result in correct performance during in situ assessments for all participants (e.g., Himle et al., 2004a, b, Miltenberger et al., 2004; Novotny et al., 2020). Researchers have demonstrated that BST can be enhanced with IST. To conduct IST, following an in situ assessment in which the child does not engage in the safety skills, a trainer intervenes and conducts training with the child until they perform the correct safety responses (Gross et al., 2007; Gunby et al., 2010; Himle et al., 2004a, b; Johnson et al., 2006; Kelso et al., 2007; Miltenberger et al., 1999, 2004; Tarasenko et al., 2010). This training consists of multiple rehearsals of the safety skills with feedback in the context where the abduction lure was delivered during the in situ assessment. The child continues to rehearse the skills until they consistently perform the safety responses.

The skills needed for children to respond safely to lures delivered in an online gaming context are similar to the skills needed to respond safely to abduction lures delivered in a face-to-face context. The three types of safety responses described in abduction-prevention research are for children to say no or otherwise refuse the request delivered by the stranger (the lure), leave the situation to get away from the stranger, and report the lure to a responsible adult (Johnson et al., 2005; Miltenberger & Olsen, 1996; Poche et al., 1988). In the online gaming context, a safe response to a lure would be similar; to recognize the lure and not respond, leave the game, and find a parent and report the lure.

Although no research has been conducted on the use of BST to teach online safety skills (e.g., safety responses to lures presented over the internet), we believe BST would be effective for teaching online gaming safety skills due to their parallel to abduction prevention safety responses. Furthermore, due to the online environment of internet gaming, we believe BST conducted over online video conferencing would be appropriate and acceptable to parents. Due to the prevalence of online gaming and the potential risk involved related to strangers luring children through the internet, it is important to teach children how to respond safely in such contexts. Therefore, the purpose of this study was to evaluate remote BST and caregiver-implemented IST to teach children online gaming safety skills.

Method

Participants and Setting

We recruited two typically developing boys and their mothers. Clark was 9 years old and Alex was 7 years old. Both participants lived in Florida. The participants did not have an intellectual disability based on parent report. Participants were recruited because they had previously used a smart phone, tablet, or computer to play the online video game “Among Us.” Both participants’ caregivers reported their child chatted with other players while playing online video games using text chat and had experience playing online with friends and strangers. Both caregivers reported that, to their knowledge, their child had not been lured by a stranger online in the past but were still concerned with the possibility of it occurring. Caregivers verbally verified their child’s reading level and typing skills to confirm their child was able to read and send messages using text chat while playing an online video game. We obtained caregivers’ permission for the participants to speak with other unknown players online (i.e., research assistants [RAs]) as part of the study and ensured the “free chat” setting was enabled on their child’s game account so they could use text chat to type messages to other players. We recruited the parent–child dyads by posting a study flyer on social media. After caregivers signed the consent form, we obtained verbal assent from each participant. During this verbal assent meeting, we did not tell the participants the true purpose of the study as this would have affected their responding; we told the participants the purpose of the study was to observe how children interacted with other players while playing online video games. The participants were allowed to play Among Us outside of research sessions to ensure the effectiveness of the deception. We debriefed the parent–child dyads 3 days after the completion of the final follow-up assessment. During the debriefing process, we explained the true purpose of the study to the participants (i.e., to assess the effectiveness of BST for teaching children how to safely respond to lures from strangers while playing online video games); we also explained they had not been playing with true strangers and the participant was never in danger while participating in the study. We emphasized the study was completely finished and if the participant encountered any lures in the future, they would be real and he should engage in the safety skills he learned in this study. After the debriefing, we asked the participants for their permission to keep their data; both participants and their caregivers gave the researcher permission to keep their data.

All training and assessment sessions occurred remotely while participants were at home with their caregiver present. We conducted the training session over the online video conferencing application Microsoft Teams. In situ assessments took place virtually in the Among Us game “lobby,” which is an online room where a player’s character waits for others to join before the start of a game and between games. The first author and RAs played games of Among Us with participants between and during in-situ assessments.

Materials

The researchers and participants had access to a computer with internet for remote training sessions; the participants and researchers had access to either a smart phone, tablet, or computer with connection to the internet to access Among Us. Among Us is a game in which players are split into crewmates and imposters, and the crewmates must work together to identify the imposters. The imposters’ goal is to eliminate all the crewmates without getting caught. Among Us is free for smart phones and tablets, and costs $5 to play on a computer. For use in the intervention, we developed a training video that modeled the safety skills for participants. To train the caregivers to respond correctly during assessments and intervention sessions, we developed a video modeling the implementation of IST and the responses to participants’ behaviors during assessments. We developed task analyses for remote BST and caregiver-implemented IST and utilized these tasks analyses to collect treatment integrity data and provided them to caregivers as part of teaching them to participate in remote BST and IST.

During in situ assessments, caregivers used a smart phone to call the researcher for the purpose of collecting data (including interobserver agreement [IOA] data) on the participants’ behavior and IST treatment integrity data. We utilized electronic data sheets or printed datasheets with a writing utensil to record the participants’ responses and calculate IOA. We provided caregivers with abbreviated social validity and side effects questionnaires weekly (see Appendix A) and full social validity and side effects questionnaires after the completion of the study (see Appendix B).

Target Behaviors and Data Collection

The target responses consisted of avoiding, escaping, and reporting in response to a lure from a stranger in the game requesting personal information. The avoidance response was not responding to the lure presented by a confederate. The escape response was exiting from the game lobby within 30 s of the lure being presented. The reporting response was telling a caregiver about the lure within 20 s of exiting the game lobby. We scored the target responses on a 4-point scale. A 0 indicated a participant responded to the lure in the chat. Any response to the confederate equated a 0, regardless of whether the response contained personal information related to the lure. A 1 indicated a participant did not respond to the lure in the chat but did not exit the game lobby within 30 s of the lure being presented. A 2 indicated a participant did not respond to the lure in the chat, exited the game lobby within 30 s of the lure being presented, but did not tell their caregiver about the lure within 20 s after exiting the lobby. A 3 indicated a participant did not respond to the lure in the chat, exited the game lobby within 30 s of the lure being presented, and informed their caregiver about the lure within 20 s of exiting the lobby.

Assessment

To establish rapport with participants and so lures during in situ assessments seemed believable, “game sessions” with the researchers occurred before and after training and did not always include the presentation of lures. A game or “round” in Among Us lasts an average of 7 to 15 min. Each game session with a participant included three to four rounds, lasting 30 min to 1 hr. Opportunities for the researchers to present a lure occurred in the game lobby between rounds; we decided that to make the game sessions seem believable, the ratio of total number of opportunities when lures were presented to the total number of opportunities when lures were not presented would be 1:3. In this way, participants would not come to expect a lure when they played the game.

To begin a game session, we created a private lobby in Among Us, obtained a code to invite other players to the game lobby, and gave the code to the RAs so there were at least four players in the game lobby. The RAs changed their username and appearance before each game session so participants believed they were playing with random players, besides the first author, each game session. Once the researchers were ready, the first author texted the private lobby code to the caregiver so the participant could join the game. When playing the game, the researchers let the participant’s team win at least half the time so that playing with the researchers became highly reinforcing. When in the lobby and in the game, the researchers provided minimal chatting and praise to the participant and to one another, so participants contacted safe, reinforcing interactions. The researchers provided at least three positive interactions per game to another player (e.g., “nice job,” “good catch”).

During a game session with an in situ assessment, we presented a lure without the participant’s knowledge that an assessment was occurring. When the group was in the lobby waiting for the game to start, one of the RAs typed a lure into the chat box directed at the participant. The lure was directed at the participant by using their character’s username or color. The lures asked for personal information, such as where a participant lived or how old he was. A full list of lures is included in Appendix C; lures were chosen at random from the list during in situ assessments. Once a lure was used, it was not repeated with the same child participant. The researchers did not provide any feedback or consequences contingent on a participant’s responses. The game session continued as usual if a participant did not engage in the target responses; the confederate typed that they had to leave and exited the game if a participant responded or did not leave the game within 30 s of the lure being presented. All players ignored the participant’s message in the chat if he provided personal information or responded to the lure. If the participant reported the lure to their caregiver, their caregiver thanked them for reporting the lure. We trained caregivers to provide consistent praise (i.e., “Thank you for telling me. You did the right thing by not talking to strangers online. That’s the best way to stay safe.”) to their child following correct responding. If a participant exited the game, we texted their caregiver to ask if their child wanted to continue playing; we did not mention the lure to the participant and all RAs changed their usernames and appearances before joining the new game lobby.

During assessments, we recorded whether the participant responded to the lure and whether he left the game. The participant’s mother was on her phone with the first author and secondary observer during the assessment so they could hear the participant’s reporting response.

Interobserver Agreement

We assigned an RA as a secondary observer to collect data during 100% of in situ assessments. The caregivers were not responsible for recording data. Before a lure was presented, we created a conference call with a secondary observer and a caregiver so the reporting response could be heard as it occurred. We let the caregiver know when the lure was about to be presented so she could be prepared to deliver the consequence (i.e., praise or in situ training depending on the experimental condition). Once the RA presented a lure, the data collectors recorded data on the participant’s safety responses. The data collectors recorded the specific lure that was used on the data sheet so it would not be used again. We recorded an agreement if both observers indicated that a response did or did not occur. A disagreement was recorded if one observer indicated that the behavior did not occur while the other observer indicated that it did. We calculated IOA by dividing the number of agreements on the three target responses by the sum of the agreements and disagreements and multiplying the result by 100. IOA was 100% across all assessments.

Treatment Integrity

We calculated treatment integrity data for 100% of BST and IST sessions; a secondary observer recorded treatment integrity data on the researcher’s implementation of BST and the researcher recorded treatment integrity data on caregivers’ implementation of IST. We developed and utilized a task analysis with yes or no questions to assess treatment integrity. The observer recorded whether a step in the task analysis occurred by checking yes, no, or not applicable (i.e., N/A) boxes on the data sheet. We calculated treatment integrity by dividing the number of correct training behaviors exhibited in a training session (i.e., the number of boxes checked “yes”) by the total number of correct behaviors possible (i.e., the sum of boxes checked “yes” and “no”). Treatment integrity was 100% for the researcher’s implementation of BST for both participants, and 90% for Clark’s mother’s implementation of IST.

Social Validity and Side Effects Questionnaire

We emailed caregivers a social validity and side effects questionnaire at the end of the study. The questionnaire included items regarding their satisfaction with their child’s participation in the study and any negative side effects they might have experienced. The questionnaire was a modified version of the questionnaire provided in Johnson et al. (2005). We emailed abbreviated side effects questionnaires to caregivers weekly. The questionnaire included three questions regarding any side effects their child may be experiencing following their participation in the assessments and game sessions that week, and one open-ended question where caregivers could include written feedback on their child’s behavior or comments if desired.

Experimental Design and Procedures

We used a nonconcurrent multiple baseline design across two participants. We conducted in situ assessments during baseline, post-BST, and at the 2-week follow-up.

Baseline

We collected baseline data for participants’ responses to lures but did not deliver feedback or consequences contingent on a participants’ responses during assessments. We instructed the parent to thank the participant if they reported the lure, but no reporting occurred in baseline.

Remote Behavioral Skills Training

We utilized BST to teach participants online gaming safety skills. The researcher first invited a participant and his caregiver to an online video conference. The researcher added an RA to the video conference call so they could record treatment integrity data. Once the participant and his caregiver joined the call, the researcher said the following statement: “Hello! How are you doing? I wanted to meet and talk about playing video games online. We are going to talk about how to stay safe when playing online video games. First, we are going to talk about how to be safe, then I’ll show you a video, and then we will practice.” The researcher began remote BST by providing the participant with information on the types of lures that could occur when online (with numerous examples) and instructions on the three safety responses (i.e., ignore the lure, exit the game, and tell an adult). The researcher explained the importance of these responses. For the modeling component, the researcher shared the modeling video with participants and their caregivers. The video consisted of a screen recording showing a child playing Among Us and a stranger presenting a lure. The player in the video did not respond to the lure and exited the game immediately. The video then showed the child getting up from their device and telling their parent about the lure and the parent thanking the child. The video, narrated by the researcher, showed examples of the child responding safely to three different lures. For the rehearsal component, the researcher prompted the participant to join a private Among Us lobby with the researcher and RA. The researcher began the role play by telling the participant to pretend as if the researcher was a stranger and to perform the safety skills; the researcher then presented a lure. If the participant exhibited all the target responses, the researcher provided descriptive praise. Caregivers praised their child’s reporting response. If a participant did not exhibit all the target responses or engaged in incorrect responses, the researcher provided corrective feedback consisting of further instruction to prompt the participant to engage in the correct responses. The researcher continued to rehearse the responses with the participant until he correctly performed all safety responses during three consecutive role plays.

We conducted an in situ assessment 1 to 3 days after the completion of the first remote BST session. If participants engaged in the target behaviors and scored a 3, we conducted another in situ assessment in 1 to 3 days. We conducted in situ assessments until participants reached the mastery criterion of three consecutive scores of 3. If participants scored less than a 3 during an in situ assessment, the researcher would initiate a booster session, but no booster sessions were needed.

Caregiver-Implemented In Situ Training

The researcher provided caregivers with detailed instructions (including a task analysis) and a modeling video showing how to implement IST. The researcher asked caregivers to watch the video and then had a virtual meeting with caregivers to discuss IST. During an in situ assessment where IST may have been necessary, caregivers remained in the same room or neighboring room as the participants and pretended to be busy (e.g., cleaning, laundry, on the phone). This was so caregivers would be able to “witness” their child’s incorrect responses if they occurred. While on the phone with a caregiver, the researcher told her the moment the lure was presented, along with a description of the participant’s responses. Caregivers waited 20 s and if the participant did not report the lure, they approached the participant and immediately implemented IST. Caregivers also implemented IST if participants reported the lure but responded to the lure and/or failed to leave the game. They looked at their child’s screen and pointed out the lure before initiating IST. They reminded their child of the three safety responses and then had their child rehearse the safety skills. They prompted their child to go back to their gaming device and pretend a lure was presented to them and then walked out of the room the participant was in so he could also rehearse the reporting response. The participant rehearsed ignoring a lure, exiting the game, and running to tell his mother about the lure. If the participant engaged in all the correct responses, his mother provided praise. If the participant responded incorrectly, his mother provided corrective feedback and modeled the correct responses. The participant rehearsed the responses with his mother until he engaged in all three correct responses three times in a row.

Follow-Up

We conducted follow-up in situ assessments 2 weeks after completion of training and assessments to assess maintenance of the online safety skills. Caregivers conducted IST if participants did not engage in all three target responses.

Results

Figure 1 shows the participants’ responses across the conditions of the study. During baseline, Clark scored 1s and 0s. Following the implementation of BST, Clark met mastery criterion by scoring three consecutive 3s. During the 2-week follow-up, Clark scored a 1; his mother implemented IST immediately following the incorrect responding. Another follow-up assessment was conducted 3 days later and Clark scored a 3. During baseline, Alex scored 1s and 0s as well. Following the implementation of BST, Alex reached mastery criterion by scoring three 3s in a row. Alex scored a 3 on his 2-week follow-up. The total durations of the remote training sessions were 24 min and 20 min for Clark and Alex, respectively.

Fig. 1.

Online safety skill scores for In Situ assessments during Baseline, Remote BST, and Follow-up

The results of Clark’s weekly and final social validity and side effects questionnaires indicated there were no negative side effects due to his participation in the study. Clark’s mother reported that she strongly disagreed that Clark was now more likely to refuse to play games online following the study, and said he is “much better equipped with the tools to know how to avoid potentially dangerous situations.” She disagreed that Clark is now more hesitant to play games online without a trusted adult present and said he “knows what to do without his parents present.” She strongly agreed that Clark is now more likely to express concerns about the safety of online games, strangers online, and personal safety, clarifying that “he told me about things in the past but with this study he became more aware and would tell me even quicker than before.” Although on the questionnaire she agreed with the statement that Clark is now more likely to talk about his fear of strangers outside of online games, she said, “he was never fearful before and during the study,” “he responded maturely without fear,” and “I know he will continue this way in the future as well.” When asked to describe other changes in her child’s behavior, Clark’s mother wrote he is “more observant of conversations in the game chat, specifically noticing if someone was asking him a personal question/talking to him.” Clark’s mother reported she was very pleased her son participated in the study, reported she was very satisfied with the way the researchers communicated what was going on throughout the study, and reported she felt very comfortable with her child playing games online with strangers after his involvement in the study.

The results of Alex’s weekly and final social validity questionnaires were similar to Clark’s results in that there were no negative side effects due to his participation in the study. Alex’s mother reported she strongly disagreed her child is now more likely to refuse to play games online compared to before the study. She reported she strongly disagreed her child is now more hesitant to play online without a trusted adult present compared to before this study. Alex’s mother strongly disagreed that he is now more likely to express concerns about the safety of online games, strangers online, and personal safety compared to before the study. She also strongly disagreed her child is now more likely to talk about his fear of strangers outside of online games compared to before this study. Alex’s mother reported “I didn’t note changes in his behavior with respect to games other than learning to stop playing and tell me if someone was asking him personal questions online.” Alex’s mother reported she was very pleased her son participated in the study, reported she was very satisfied with the way the researchers communicated what was going on throughout the study, and reported she felt very comfortable with her child playing games online with strangers after his involvement in the study.

Discussion

The results of the present study illustrate the effectiveness of remote BST and caregiver-conducted IST for teaching online gaming safety skills to two participants. These findings are consistent with results of research evaluating the use of BST and IST to train abduction prevention skills and promote generalization to in situ assessments (Johnston et al., 2006), as well as research demonstrating the effectiveness of caregiver-implemented IST (Gross et al., 2007). This is the first study to examine the potential of utilizing BST to teach safety skills pertaining to online video games, and the results show the effectiveness of BST and the potential for future research in this environment.

A strength of this study was the remote nature of the training and assessments. The researcher never met with participants, their caregivers, or the RAs in person. The researcher trained the RAs remotely, met with participants remotely online during game sessions, and spoke with caregivers remotely over the phone or on video conference apps for training or data collection. This highlights the efficiency, contextual fit, and safety of the training program, especially in a time where the danger of infecting others with COVID-19 is still present. The remote nature of the study had other benefits as well. Scheduling training or game sessions was less challenging because the researcher and RAs could participate in the study from any setting instead of having to meet at a predetermined location; there were also fewer time constraints because of the increased flexibility for meeting times. Participants and their caregivers participated in training and game sessions from their homes; this was also beneficial due to the number of meetings that was needed to complete the study. When there were not enough RAs available to play in a game session, the researcher or an RA could play Among US on two or more devices at the same time to add more players to the game lobby.

This study provides a model for the development and implementation of in situ assessments for online safety skills. The researcher took several steps to ensure the deception was successful by randomizing as many variables as possible in the game lobbies to make the child participant believe he was playing with strangers. The RAs changed their name and appearance before they initially entered the game lobby for a game session; if an RA was disconnected from the lobby, they changed their name and appearance again to make it seem like they were a new stranger. Each game session, the researcher varied the number of players that were in the game lobby; the number of players changed throughout game sessions, such as an RA leaving the game to decrease the number or joining the game to add a player. The researcher also varied the number of players who were in the lobby when participants joined for a game session. This randomization of variables closely resembled the environment of online video games in everyday life where the number of players can change frequently for a variety of reasons, thus making it unlikely participants would sense that the game was staged as part of the experiment. After probe 5, Clark’s mother told the researcher that Clark made a comment that the pink player always seemed suspicious. To avoid the possibility of Clark realizing he was not playing with true strangers, the researcher began to systematically vary the manner the RAs wrote messages in the lobby across game sessions. For example, RAs wrote their messages in predetermined manners during a game session, such as typing in all capital letters, all lowercase letters, using punctuation or no punctuation, using certain phrases or slang, abbreviated words, etc. After these changes were implemented, Clark did not make any other comment about a specific character’s behavior.

Despite the success of the training, there were several limitations that should be noted. First, there were only two participants in the multiple baseline design across participants. Although experimental control was demonstrated, more participants would have resulted in a stronger demonstration of experimental control.

Second, is it is not known if Clark’s safety skills score of 1 was a valid score for the 2-week follow-up assessment because it is possible Clark did not see the lure. When other lures were presented, the confederate first interacted with Clark in the game context to make sure he was paying attention to the chat before presenting the lure. However, in this assessment, due to technical difficulties, 15 s elapsed between game-related chat and the presentation of the lure. When Clark did not engage in the safety skills, the researcher prompted Clark’s mother to implement IST; she approached him and told him that the researcher told her a stranger asked him a personal question. Clark responded that a stranger did not ask him anything, but then he checked his game (as evidenced by his verbal behavior) and immediately left the game. Clark told his mother that he did not see the lure because the game does not give notifications when messages are sent. Nonetheless, Clark’s mother continued to implement IST as the researcher listened remotely and Clark scored a 3 during the next follow-up probe.

Third, the researchers do not know if Clark or Alex believed they were playing with strangers during all game sessions. Both participants made comments about the RAs during the debriefing. When the researcher explained the true purpose of the study to Clark and that Clark had not been playing with real strangers or presented with real lures, Clark responded, “I had a feeling.” Despite this response, Clark’s behavior throughout the study did not lead the researcher to believe that he knew of the deception. Clark played each game similarly and spoke with other players the same way each time, he never asked his mother or the researcher questions about the study, and his behavior stayed the same following training other than exhibiting the correct safety responses. During Alex’s debriefing, Alex had a confused look on his face when the researcher explained the true purpose of the study. Alex made a comment that he had seen the same “stranger” twice. The researcher explained the RAs were instructed to change their names and appearances every time they joined, and when the researcher asked Alex to elaborate, he said he thought the other players were different people each time. It is unclear if Alex was confused or if he knew he was playing with the same people during each game session. Both Clark and Alex stated they understood why deception was necessary in the study and that they were okay with it.

Another potential limitation was the researcher’s presence during game sessions. This may have caused reactivity that affected the participants’ behavior and may have also further weakened the deception, particularly after BST was implemented. However, the researcher needed to be in the games because he created the game lobbies and sent the private codes to caregivers and the RAs. Future researchers should seek to keep the researcher out of the game. Lastly, there was only so much the researchers could do to vary the number of players in the game. At least 4 players were needed to play the game and there were not enough RAs to have more than 10 players in the lobby at one time (including the researcher and one RA on two devices at the same time).

There are several potential extensions for future research pertaining to BST and online gaming safety skills. Many different games can be used, particularly online video games where players speak with one another using a microphone. Researchers would need to solve potential challenges that would surface by using voice chat, such as having different confederates with distinctly different voices or by using a voice changer. In some games, players can message other players privately or add players as friends; both of these variables could be included in future studies to teach children how to safely respond to lures presented in these ways. In the future, the study can be completed with a video game that has a smaller lobby so not as many RAs are needed at one time. Popular games such as Fortnite and Call of Duty: Warzone have 100 player lobbies, but teams of 2 to 4 players talk with each other and other players cannot hear what is said. This would protect the participant in case they give away personal information and minimize the number of RAs needed to serve as confederates. Researchers should analyze and systematically manipulate the ratio of lure presentations during game sessions to learn which ratio closely matches real life circumstances. The ratio of the presentation of lures was at most 1:3, but it is unknown if this was the best ratio to maintain the deception and make in situ assessments seem more realistic.

Future researchers should vary the types of lures used in assessments, such as those described by Miltenberger and Olsen (1996). It would be valuable to specifically test the effectiveness of BST for training safety responses to incentive lures because in most online video games today, there is the ability to trade items with players that are purchased with real money or in-game currency (e.g., ROBLOX ). It would be interesting to see how participants react to lures that are presented with the offer of in-game currency, cosmetics, or special items. Future researchers could also train different behaviors that are functionally equivalent to the avoidance, escape, and reporting responses. The escape response taught in this study may not generalize to other video games that have longer matches or more difficulty to find lobbies to play in. Also, the reporting response taught in this study may not maintain over time if parents begin to limit their child’s access to online games due to fear of strangers’ lures. Instead of exiting a game completely or reporting the lure to a caregiver, the responses could be to “kick” the confederate from the lobby (if the participant is in charge of the lobby), block the confederate, or report their account. These three behaviors require less response effort and are closer to what a player may do in a real-life scenario.

Research should extend to different demographics, such as children or young adults with autism or other developmental disabilities who are at risk of lures on online video games. Future researchers could have peers provide BST to participants, implement BST for a group of participants, or have parents or teachers implement BST. In the current study, participants may have thought it was strange that lures were only directed towards them. In future research, confederates could direct lures at other RAs and the RAs could model correct or incorrect responses for the participant. Overall, this study provides a promising introduction into the intersection of the online world and applied behavior analysis, offers a model for assessing safety skills in the online environment, and suggests BST may be an effective approach for training online safety skills.

Appendix A

Abbreviated Side Effects and Social Validity Questionnaire

Instructions: Please circle, highlight, or write your answers

1. After this week, my child is now more likely to refuse to play games online.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly Agree

• 2.After this week, my child is now more hesitant to play online without a trusted adult present.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly Agree

• 3.After this week, my child is now more likely to talk about their fear of strangers outside of online games.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly Agree

• 4.If there were any changes in your child’s behaviors, or you have additional comments, please include them here:

Appendix B

Side Effects and Social Validity Questionnaire

Instructions: Please circle, highlight, or write your answers

1. Compared to before this study, my child is now more likely to refuse to play games online.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly Agree

If a change occurred, please describe briefly:

• 2.Compared to before this study, my child is now more hesitant to play online without a trusted adult present.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly Agree

If a change occurred, please describe briefly:

• 3.Compared to before this study, my child is now more likely to express concerns about the safety of online games, strangers online, and personal safety.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly Agree

If a change occurred, please describe briefly:

• 4.Compared to before this study, my child is now more likely to talk about their fear of strangers outside of online games.

• Strongly disagree

• Disagree

• Neutral

• Agree

• Strongly Agree

If a change occurred, please describe briefly:

• 5.Other changes I noted in my child’s behavior are:

Please describe or mark N/A if no change was observed

• 6.How pleased are you that your child participated in the study?

• Very Pleased

• Pleased

• Neutral

• Unpleased

• Very Unpleased

• 7.How satisfied are you with the way the researchers communicated what was going on throughout the study?

• Very satisfied

• Satisfied

• Neutral

• Unsatisfied

• Very Satisfied

• 8.Do you feel more comfortable with your child playing online with strangers after their involvement in this study?

• Very comfortable

• Comfortable

• No change

• Uncomfortable

• Very uncomfortable

• 9.Did you terminate your child’s participation in the study?

• Yes

• No

If yes, please explain why:

• 10.Please note any additional comments you have about the study.

Appendix C

Table of potential lures

• Age questions

  • How old are you?
  • When is your birthday?
  • What month were you born in?
  • What day were you born on?
  • What year were you born in?
  • Are you in middle school?
  • Are you in elementary school?
  • Are you in X grade, too (depends on grade of participant)?
• Gender questions

  • Are you a boy or girl?
  • Are you a male or female?
  • M or F?
  • Are you a x, too (depends on gender of participant)?
• Location questions

  • Where do you go to school?
  • Where do you live?
  • Where are you from?
  • What state are you in?
  • Are you from the USA?
  • Do you go to x location ever (near the school they go to)?
  • Do you live in Florida, too?
• Personal information

  • What is your name?
  • What is your full name?
  • What is your last name?
  • What’s your phone number?
  • What’s your parents phone number?
  • What is your mom’s name?
  • What is your dad’s name?
  • Do you have social media?
  • Do you have Snapchat?
  • What’s your Instagram?
  • Do you have Facebook?

Authors' Contributions

All authors contributed to conceptualization and writing.

Funding

There was no funding for this research.

Data Availability

Data will be made available on request.

Code Availability

Not applicable.

Declarations

Conflicts of Interest/Competing Interests

None.

Ethics Approval

Study approved by the university Institutional Review Board.

Consent to Participate

Both participants provided consent.

Consent for Publication

All authors consent to publishing this article.