Implementing Virtual Interview Training for Transition-Age Youth: Practical Strategies for Educators

Abstract

This column shares strategies for implementing a technology-based intervention within secondary special education transition services. Virtual Interview Training for Transition-Age Youth is a web-based job interview training that uses educational content and virtual hiring managers (Rita and Travis) to help teach job interview skills. Eight strategies were assembled by research staff, teachers, and administrators during two research studies conducted across n=47 schools that delivered the program (commonly called “Rita and Travis”) to n=561 students in secondary special education transition programs. Eight support strategies for implementing Rita and Travis are described. Strategies include: prepare for implementation, ensure administrative support, prioritize training and support, address logistical challenges, integrate the program into the curriculum, take advantage of the administrative center, individualize student support, and offer virtual delivery. The purpose of this column is to prepare others for successful implementation of Virtual Interview Training for Transition-Age Youth within special education transition programs.

Introduction

Special education transition services and pre-employment transition services (Pre-ETS), as required by the Individuals with Disabilities Education Improvement Act (IDEIA, 2004) and the Workforce Innovation and Opportunity Act (WIOA, 2014), respectively, provide a critical opportunity for employment-focused intervention. Although there are a number of modalities through which to deliver Pre-ETS, technology-based interventions are an increasingly pervasive modality for delivery (Grynszpan et al., 2014; Odom et al., 2015). Technology has shown to be especially useful for transition-age youth (Zionch, 2011), particularly in the area of job interview skills.

A recent scoping review indicated several technology-based interventions focused on job interviews for autistic youth (Kim et al., 2022). One research group used video modeling with a group of n=8 students using a mobile device and found that video modeling was preliminarily effective in teaching interview skills (Hayes et al., 2015). Another research group had participants in two studies (n=7 and n=13) complete mock job interviews with a robot while measuring cortisol levels (Kumazaki et al., 2017, 2019). The android robot was life-size and resembled a real human and spoke pre-recorded responses with the press of a button by a live operator. Results from these studies showed improved self-confidence and lower cortisol levels after practicing interviews. In addition, Burke et al. (2018, 2021) developed Virtual Interactive Training Agents (ViTA) to teach job interview skills to n=32 adults with autism and n=153 adults with autism or other developmental disabilities, respectively. This intervention demonstrated preliminary effectiveness at teaching job interview skills and improving job interview self-efficacy, respectively. However, some limitations of the ViTA model includes the use of three cameras, a large screen television, a private room, and a live operator. Additionally, these studies were not rigorously evaluated using randomized controlled trials and were not associated with greater access to real-world functional outcomes such as competitive employment. Strickland et al. (2013) conducted a randomized study with n=22 transition-age youth with autism in which n=11 youth utilized the JobTIPS intervention. JobTIPS is a web-based program consisting of educational content and a virtual interview simulation. The results showed that participants improved their verbal responses and had limited improvement in the delivery of their responses (i.e., posture, eye contact).

Recently, Virtual Interview Training for Transition-Age Youth (commonly referenced as “Rita and Travis” as these names reflect the virtual hiring managers within the tool) was used in two recent studies (1 RCT [n=71] and 1 quasi-experimental [n=356]), which observed the initial effectiveness among n=427 transition-age youth with disabilities and initial implementation feasibility, acceptability, and usability across 37 schools (Smith et al., 2021; Smith, Sherwood, et al., 2022; Sherwood et al., in press). Notably, Rita and Travis improved job interview skills, lowered job interview anxiety, and resulted in better competitive employment outcomes across both studies. In addition to being versatile in terms of how and where the intervention is delivered, teachers were able to deliver the intervention to multiple students at once, with each student working on their own computer or tablet or following along in whole-group instruction. Due to its stronger research base and our direct evaluation of this intervention, this column shares eight strategies for implementing Rita and Travis within secondary transition services.

Virtual Interview Training for Transition-Age Youth (Rita and Travis)

Virtual Interview Training for Transition-Age Youth (also referred to as Rita and Travis) is an internet-based job interview simulator (licensed by SIMmersion LLC; https://jobinterviewtraining.simmersion.com/) that uses educational content, two virtual hiring managers (Figure 1; Rita Muniz and Travis Bishop, who are automated using video clips through a probability-based algorithm), and an immersive job coach (Kendra) to teach job interview skills. Students speak with one of the virtual hiring managers and answer their questions about 1 of 14 jobs (either by speaking into a microphone or by clicking on-screen text). The program has three difficulty levels (easy, medium, and hard) and targets ten job interview skills: confidence, positivity, professionalism, interest in the job, honesty, dependability, ability to work well with others, sharing strengths and skills, sharing past experiences, and sharing limitations. Notably, Rita and Travis is an adaptation of Virtual Reality Job Interview Training (VR-JIT; a web-based job interview simulator designed for adults with serious mental illness) and was tailored by partnering with 45 stakeholders from the autism community to specifically meet the needs of transition-age youth with autism and other disabilities (Smith et al., 2020).

Figure 1.

Virtual Interviewer Travis Bishop

Regardless of an intervention’s effectiveness, implementation with failed fidelity is often the underlying reason for an intervention’s lack of impact on person-level and organization-level outcomes (Klein & Knight, 2005). In order to enhance implementation within Pre-ETS, this column shares eight practical strategies learned from preparing more than 60 teachers at 37 schools to implement Rita and Travis (Smith et al., 2021; Smith, Sherwood, et al., 2022; Sherwood et al., in press). While these strategies have not been measured systematically, they are intended to enhance the fidelity of implementation. Across two prior published studies, Rita and Travis was delivered to n=427 of students receiving Pre-ETS in schools. In addition, Rita and Travis initial implementation outcomes were evaluated by our team in an additional 10 schools (including 134 students). Overall, our evaluations have been across 47 schools and 561 students with disabilities The disability category breakdown of students was n=155 students with autism, n=150 students with a learning disability, n=52 students with an emotional disability, n=97 students with an intellectual disability, n=61 students with other health impairment, n=3 students with traumatic brain injury, n=38 with other disabilities, and n=5 students with missing disability category data. Students were enrolled in a combination of public, private, or charter high schools, secondary transition programs, and a small subsample was enrolled in a virtual academy. The purpose of this column is to share lessons learned to prepare others for successful implementation of this technology-based intervention.

Practical Strategies for Implementing Rita and Travis

Strategy #1 – Prepare for Implementation

Prior to implementing any new intervention, it is recommended that administrators, teachers, and staff gather data to evaluate the potential roadblocks to Rita and Travis implementation, strengths and weaknesses of the school, and leadership and school climate opportunities and constraints. Administrators may benefit from learning about the pre-implementation preparation activities (e.g., organizing a workflow, training teachers to become implementers; testing program on school devices) required to prepare their school for implementation. Schools may also need to consider de-implementing (i.e., removing, replacing, reducing) other interventions that are not working (e.g., logistical problems or ineffective) in order to make time and space for new interventions (Prusaczyk, 2020).

Strategy #2 - Ensure Administrative Support

Prior to purchasing any intervention or product, school administrators should ask to see scientific evidence on the intervention’s effectiveness. “For example, there are at least 16 scientific publications supporting the scientific development, initial effectiveness and implementation process outcomes of Rita and Travis (and the original version, VR-JIT) at improving job interview skills and access to competitive employment for youth and adults with disabilities (e.g., Sherwood et al., in press; Smith, et al., 2015; Smith, et al., 2014; Smith et al., 2020; Smith, Sherwood, et al., 2022; Smith et al., 2021; Smith, Smith, et al., 2022; Ustel et al., 2021). This information should then be communicated to teachers and the rest of the implementation team, so they are knowledgeable about the research base. Informational sessions introducing Rita and Travis should be held at each school prior to implementation. Informational sessions open to teachers, staff, parents, and school board members may result in more school and community buy-in and engagement. Additionally, holding orientation sessions with administrators and teachers all in the same room may increase communication, collaboration, and buy-in. This can help connect the whole staff and establish study needs and expectations. Further, more communication and support from administrators allow teachers to be more confident in their decision to implement the program.

Strategy #3 - Prioritize Training and Support

During training, teachers are provided an overview of Rita and Travis and a demonstration during which they simultaneously navigate the program. After the initial training, all participating teachers complete at least three virtual interviews to get more familiar with the program prior to delivering it in the classroom. In order to provide additional support, there are a series of YouTube videos which include step-by-step instructions for navigating all features of the program. These videos are publicly available and recommended for anyone planning to implement the program within their school (see Table 1). The e-learning material is designed to supplement or replace existing interview preparation curriculum and give teachers a better understanding of how virtual interviews are scored and how students should respond during virtual interviews. Additionally, SIMmersion’s navigational resources provide tutorials on features, such as rewinding transcripts, saving a current interview play, viewing past plays, and viewing past transcripts. Incorporating these pieces into the training of teachers can create more comfort and familiarity during the implementation process.

Table 1.

Rita and Travis Resources for Administrators, Teachers, and Staff

Resource	Description
Purchasing Rita & Travis	https://www.simmersion.com/ ($110 per license per year)
Rita & Travis Training Videos	https://youtube.com/playlist?list=PLip_V-QSugn6mq23v9-kbsvNKV9sgxafy
Rita & Travis Learning Goals	https://bit.ly/3nYoMDh
Rita & Travis Job Descriptions	https://bit.ly/3nTdBvD
Rita & Travis Training Checklist	https://bit.ly/39mFOT6

Strategy #4 – Address Logistical Challenges

When it comes to logistics, implementation can benefit from lengthier classroom time (i.e., 60–70 minutes) in order to reserve some time at the beginning of class for passing out computers, logging in, and troubleshooting. Lower student absentee rates are also beneficial as it allows for all students to navigate the e-learning portion as a class. One significant barrier to the implementation is that transition-age students are often off-site at internships or workplace experiences. Other logistical challenges include: (a) students using the program all in one room can be difficult for some students with respect to filtering out background noise, (b) some students may need frequent breaks that could be more disruptive to shorter class times, and (c) some students may have disruptive behaviors. Other interruptions could include IEP testing and meetings, altered school schedule days (e.g., half days, SAT testing), and holidays. While some of these issues are unavoidable, certain accommodations can be made to limit their frequency and impact (e.g., coordinate use on days which are not expected to be disruptive).

Strategy #5 - Integrate into the Curriculum

While some schools may have a dedicated pre-employment curriculum (e.g., Practical Assessment Exploration System [Smith-Demers et al., 2017], Project SEARCH [Rutkowski et al., 2006], Life Centered Education [Brolin, 1997]), others have homegrown programming that does not follow a specific curriculum. Implementation may be smoother at schools that have designated classroom space, a designated teacher, and designated class time (Becker & Domitrovich, 2011; Herlitz et al., 2020). For example, some schools have an existing designated space for pre-employment activities (e.g., computer lab, classroom). A designated teacher, or teachers, should deliver Rita and Travis during a time that is most natural for their Pre-ETS program. For example, having a dedicated staff person with dedicated time and buy-in, flexibility to facilitate make-up days to allow for student absences, and use of a designated computer lab rather than laptops or tablets that need retrieved and set up each time may be helpful. This consistency is beneficial for both teachers and students.

Strategy #6 - Take Advantage of the SIMmersion Administrative Center

Rita and Travis includes access to SIMmersion’s Administrative Center portal where teachers can assign students their usernames, track individual student level (or classroom-level) usage, interview scores, create reports, and review virtual interview transcripts. Reports can be customized for individual students (or for whole classes or schools) to show how many interviews they complete at each level of difficulty, their highest and lowest scores, an overall average of their scores, if they pass or fail an interview, how many minutes they spend reviewing the e-learning components, how many minutes they spend during an interview, and a complete transcript of their interviews. Teachers can include data from the Administrative Center into students’ individualized education program (IEP) as part of their progress toward meeting a goal. Reports can be automated so teachers can receive them daily, weekly, or annually.

Strategy #7 - Individualize Student Support

There is some variability in how students interact with Rita and Travis. Research staff and teachers can adjust the level of support for each student based on their ability to navigate technology, their reading level, and following the protocol (i.e., appropriately advancing levels). Also, students can be given breaks when needed and be allowed to interact with the virtual interviewers at their own pace. Schools may also provide teaching assistants, as some students require more one-on-one guidance navigating the program and remembering login information. To get the most out of virtual interview training, it is recommended that users complete at least 9 virtual interviews, (e.g., three easy, three medium, and three hard). This can be a guide for teachers to fit it into their lessons and adapt as needed to accommodate individual student needs. When using Rita and Travis, it is extremely helpful to observe each student’s interview scores. It is important to create a plan for each individual student (and follow their individualized education program) to decide how much time they need to dedicate to each level of play (i.e., easy, medium, hard) and how many virtual interviews they need to complete to reach maximum performance.

Strategy #8 – Offer Virtual Delivery

Rita and Travis can also be delivered within virtual school settings. Teachers can lead orientation sessions via an online learning management system, while students are able to access it from home. There are three main benefits to implementing the program in a virtual classroom setting: (a) Students can access the tool when it is most convenient for them, (b) Students can complete interviews in an environment that suits them best, and (c) Students have the opportunity to complete interviews without the added pressure of their teacher and peers observing them. Although there are advantages of virtual delivery, there are some challenges with this method: (a) It may be more difficult to individually support each student and their independent learning style and accommodation needs, (b) Without the tangible presence of an instructor, navigating and troubleshooting might lead to difficulties in using the program, and (c) Incentivizing and encouraging students to use the program may be more difficult to do virtually; though teachers can use the administrative center to monitor usage and encourage increased use of the tool.

Conclusion

Transition services and Pre-ETS are an integral part of special education for transition-age youth. Rita and Travis is a technology-based intervention with a growing research base that focuses on developing job interview skills through an internet-based job interview simulator with automated feedback. This column reported eight strategies to help inform future implementation of Rita and Travis within school-based Pre-ETS. The strategies include: prepare for implementation, ensure administrative support, prioritize training and support, address logistical challenges, integrate the program into the curriculum, take advantage of the administrative center, individualize student support, and offer virtual delivery. In addition, this column includes a list of resources for teachers and administrators (see Table 1) to assist with implementation. If applied, these strategies will enhance the implementation fidelity of Rita and Travis within special education transition programs.