Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Oct 1.
Published in final edited form as: J Autism Dev Disord. 2015 Oct;45(10):3364–3369. doi: 10.1007/s10803-015-2470-1

Brief Report: Vocational Outcomes for Young Adults with Autism Spectrum Disorders at Six Months After Virtual Reality Job Interview Training

Matthew J Smith 1,✉, Michael F Fleming 1,2, Michael A Wright 1, Molly Losh 1,3, Laura Boteler Humm 4, Dale Olsen 4, Morris D Bell 5
PMCID: PMC4772401  NIHMSID: NIHMS761931  PMID: 25986176

Abstract

Young adults with high-functioning autism spectrum disorder (ASD) have low employment rates and job interviewing presents a critical barrier to employment for them. Results from a prior randomized controlled efficacy trial suggested virtual reality job interview training (VR-JIT) improved interviewing skills among trainees with ASD, but not controls with ASD. We conducted a brief survey with 23 of 26 participants from this study to evaluate their vocational outcomes at 6-month follow-up with a focus on whether or not they attained a competitive position (employment or competitive volunteering). Logistic regression indicated VR-JIT trainees had greater odds of attaining a competitive position than controls (OR 7.82, p < 0.05). Initial evidence suggests VR-JIT is a promising intervention that enhances vocational outcomes among young adults with high-functioning ASD.

Keywords: Autism spectrum disorder, Virtual reality training, Vocational outcomes

Introduction

High-functioning young adults with an autism spectrum disorder (ASD) have a high unemployment rate (50–75 %) (Hendricks 2010; Liptak et al. 2011; Volkmar et al. 2009) that may be partially explained by the unavailability of services to support obtaining employment (Gerhardt and Lainer 2011; Higgins et al. 2008). Thus, there is a clear need to develop novel evidence-based vocational services for these individuals (Shattuck et al. 2012b). A recent Autism Speaks stakeholder meeting identified the job interview as a major barrier to obtaining work for high-functioning adolescents and adults with ASD (Autism Speaks 2012). Specifically, individuals with ASD may have verbal and nonverbal social communication deficits that could interfere with the reciprocity and flow of conversation that occurs during an interview (Morgan et al. 2014). Hence, job interview skills may be a critical target for interventions aimed at enhancing vocational outcomes for adults with ASD by improving access to competitive employment.

In an effort to help facilitate job interview skill training for individuals with neuropsychiatric disorders, we developed and evaluated the efficacy of virtual reality job interview training (VR-JIT; a program using repetitive simulated job interviews based on hierarchical learning) in a series of randomized controlled trials (RCTs). The results suggest that adults with a psychiatric disability (primarily mood disorders) (Smith et al. 2014a; Smith et al., in press-a), and high-functioning young adults with ASD (Smith et al. 2014b) randomized to the VR-JIT group improved their interviewing skills as compared to controls. Although these studies demonstrated improved interviewing skills, the utility of the training would be further validated if VR-JIT contributed to improved vocational outcomes.

Thus, in a 6-month follow-up of the individuals with psychiatric disabilities, we observed that VR-JIT trainees reported better vocational outcomes after completing the efficacy study compared to controls (Smith et al., in press-b). In the current study, we evaluated the vocational outcomes of young adults with high functioning ASD who previously completed an efficacy study of VR-JIT. Based on our recent findings, we hypothesized that VR-JIT trainees would have greater odds of accepting an offer for a job or competitive volunteer position. We also evaluated whether neurocognition, social cognition, ASD symptom severity, and VR-JIT process measures recorded during the efficacy trial (i.e., role-play performance, number of completed VR-JIT trials, VR-JIT performance) were correlates of position acceptance among trainees.

Methods

Participants

We conducted an RCT to evaluate the efficacy of VR-JIT among n = 26 young adults (ages 18–31) with high-functioning ASD who were actively seeking employment. Participation in the efficacy study required a non-specific diagnosis on the autism spectrum and was determined using the Social Responsiveness Scale, Second Edition (Constantino and Gruber 2012) and clinical records. Participants were recruited through advertisements located at community-based service providers, local universities, community-based support groups, and online ASD support groups (e.g., Facebook). Additional details on the inclusion criteria for these participants are discussed here (Smith et al. 2014b).

Participants were re-contacted 6 months after finishing the efficacy study to complete a brief follow-up survey. Twenty-three participants (n = 15 VR-JIT trainees and n = 8 controls) completed the follow-up survey (a retention of 88.5 %). Northwestern University’s Institutional Review Board approved the study protocol, and all participants provided informed consent.

Intervention

Virtual Reality Job Interview Training (VR-JIT) is an application that is available for download from the internet and can also be installed from a DVD. The software is compatible with the two leading operating systems, and VR-JIT can be viewed on a typical computer monitor without the use of specialized glasses or headsets. SIMmersion LLC (https://www.simmersion.com) created the intervention to improve job interview skills for adults with a range of disabilities. Trainees simulated a job interview with a virtual human resources representative at a large department store. The character generates interview questions using an algorithm based on customizable features (e.g., identifying the need for accommodations) as well as the types of responses provided by the trainee that influence rapport. Images of the character and the VR-JIT interface can be found at https://www.jobinterviewtraining.net, a website designed to increase the distribution potential of VR-JIT. Please see the detailed methods on the development of VR-JIT here (Smith et al. 2014b).

Efficacy Study Measures

Below is a brief summary of variables collected during the efficacy study, and analyzed in the present study. The participants’ demographic characteristics (e.g., age, gender, race), and vocational history (e.g., months since prior employment, prior vocational training) were obtained via a self-report interview. Neurocognition was assessed using the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) (Randolph et al. 1998). Social cognition was assessed using the Bell-Lysaker Emotion Recognition Task (i.e., a measure of facial affect perception using dynamic videos) (Bell et al. 1997) and using a cognitive empathy task (i.e., a measure of emotional perspective-taking using static social scenes) (Smith et al. 2014c).

Participants rated their self-confidence at performing job interviews using a 7-point Likert scale to answer nine questions, with higher scores reflecting greater self-confidence. The internal consistency was strong for the post-test phase (α = 0.94) of the efficacy study among individuals who completed 6-month follow-up.

Process measures included (1) changes in job interview role-play performance, (2) a count of the number of VR-JIT trials completed, and (3) changes in VR-JIT performance across trials. Role-play scoring was conducted by blinded raters with expertise in human resources and based on the VR-JIT learning objectives. Change in VR-JIT performance scores were computed using a linear regression slope for each subject based on the regression of their performance scores on the log of trial number. Additional details can be found here (Smith et al. 2014b).

Six-Month Follow-Up

Participants were contacted via the phone or through email and instructed to complete a brief follow-up survey. They were not specifically encouraged to rely on caregiver support to accurately answer the questions. The survey included seven questions that asked participants to reflect on the past 6 months since their completion of the efficacy study. Specifically, they were asked: (1) How many weeks have you been looking for a job or volunteer work?; (2) How many job interviews have you completed?; (3) How many jobs have you been offered?; (4) Did you accept any of these job offers? If yes, how many?; (5) “How many volunteer interviews have you completed?”; (6) How many volunteer positions have you been offered?; and (7) Did you accept any of these volunteer offers? If yes, how many? Questions 2–3, 5–6 were recoded to reflect whether or not an interview was completed and whether or not a position was offered (coded: 0 = no, 1 = yes). A competitive position is defined as employment or competitive volunteer position as both outcomes required successful completion of an interview. Moreover, this approach is based on recent evidence that young adults with ASD attain more positive outcomes if they contribute to the community through employment or volunteer work (Henninger and Taylor 2013; Taylor 2009).

Data Analysis

Analyses of variance (ANOVA) and Chi square analyses were conducted using baseline demographic, vocational, clinical and cognitive data to examine the between-group differences [mean (SD)] at baseline among participants who completed the 6-month follow-up survey.

A logistic regression was conducted with “accepting a position” (1 = yes, 0 = no) as the dependent variable to evaluate whether or not participants who trained with VR-JIT had greater odds of obtaining a competitive position. Post-test self-confidence and prior paid employment were included as covariates based on the results from the efficacy study, which indicated that VR-JIT trainees had higher self-confidence than controls at the end of the study (m = 51.7, SD = 7.5 vs. m = 45.3, SD = 7.7; p = 0.07, Cohen’s d = 0.84) as well as higher rates of prior paid employment (trainees: 60 % vs. controls: 37.5 %). Moreover, prior paid employment (e.g., during high school) is associated with improved vocational outcomes among adults with ASD (Wehman et al. 2012, 2013). Odds ratios (OR) were generated and presented with 95 % confidence intervals (CI), while Nagelkerke R2 was computed to determine the proportion of explained variance. We generated a directional hypothesis based on our prior work demonstrating VR-JIT training is associated with improved vocational outcomes 6 months after study completion (Smith et al., in press-b) and as such, we used one-tailed tests of significance.

We explored the relationships between the dependent variables of ‘accepted a position,’ ‘weeks searching for a position’ and ‘completed an interview,’ and the independent variables of neurocognition, social cognition, and VR-JIT process measures (i.e., role-play improvement, number of completed VR-JIT trials, VR-JIT performance slope) using point-serial (for binary variables) and Pearson correlations.

Results

Participant Characteristics

We did not observe any between-group difference in baseline characteristics for participants who completed the 6-month follow-up survey (Table 1, all p < 0.10).

Table 1.

Baseline characteristics of study sample at 6-month follow-up

Control group (n = 8) VR group (n = 15) χ2/T-statistic
Demographics
  Mean age (SD) 23.1 (3.3) 25.0 (6.9) 0.7
  Gender (% male) 75.0 % 73.3 % <0.1
  Parental education, mean years (SD) 15.8 (0.5) 14.9 (2.7) −1.1
Race
    % Caucasian 37.5 % 46.7 %
    % African-American 37.5 % 26.7 % 1.8
    % Latino 25.0 % 13.3 %
    % other 0.0 % 13.3 %
Vocational history
  Months since prior employment, mean (SD) 19.3 (25.0) 32.3 (23.5) 0.8
  Prior full-time employment (%) 12.50% 13.30% <0.1
  Prior paid employment (any type) (%) 37.50% 60.00% 1.1
  Prior participation in vocational training program 25.00% 46.70% 1.1
Cognitive function
  Global neurocognition, mean (SD) 88.1 (20.3) 89.5 (22.3) 0.1
  Basic social cognition, mean (SD) 0.82 (0.11) 0.72 (0.17) −1.4
  Advanced social cognition, mean (SD) 0.78 (0.14) 0.75 (0.08) −0.6
Social Responsiveness Scale
  Social awareness 63.6 (10.8) 63.3 (6.3) −0.1
  Social cognition 63.8 (13.4) 66.7 (9.5) 0.6
  Social communication 64.1 (11.9) 68.3 (11.6) 0.8
  Social motivation 59.0 (13.8) 61.3 (9.0) <0.1
  Restricted interests and repetitive behaviors 68.1 (10.1) 70.0 (12.3) 0.4
  Total T-Score 65.1 (12.3) 68.7 (7.8) 0.7
Open in a new tab

Process Measures and Vocational Outcomes in 6-month Follow-Up Subjects

VR-JIT trainees who completed 6-month follow-up were characterized by improvements in role-play performance (mean difference = 3.57, SD = 3.54), they completed m = 16.40 (SD = 4.27) VR-JIT trials, and their VR-JIT performance score improved m = 5.31 points (SD = 5.69) for every trial they completed. Table 2 displays the between-group differences in vocational outcomes at 6-months. The control and VR-JIT groups did not differ with respect to the total number of weeks that they spent looking for a competitive position after completing the study, the total number of interviews completed, the number of interviews completed per week, or by the proportion of participants who completed an interview or received a job offer (both p > 0.10). Although the proportion of participants who accepted a position was larger in the VR-JIT group (53.3 %) than controls (25.0 %), the difference was only a trend (p = 0.09).

Table 2.

Six-month follow-up between-group differences on vocational outcomes

N Control group (n = 8) N VR-JIT group (n = 15) χ2/T-statistic
Weeks looking for a position (job or volunteer) mean, SD 16.3 (10.9) 13.5 (12.2) −0.5
Interviews completed (job or volunteer) mean, SD 1.9 (2.8) 2.1 (2.2) 0.2
% of subjects who completed an interview (job or volunteer) 5 62.50% 12 80.00% 0.8
% of subjects who received an offer (job or volunteer) 4 50.00% 9 60.00% 0.2
% of subjects who accepted a position (job or volunteer) 2 25.00% 8 53.30% 1.7+
Open in a new tab
+

p < 0.10

Odds Ratio of Accepting a Position by 6-month Follow-Up

A logistical regression model examined whether completing VR-JIT training contributed to the odds of accepting a job or competitive volunteer position (Table 3). The model’s Nagelkerke R2 was 0.212 (21.2 % variance explained). The logistic regression indicated that participants who completed VR-JIT had greater odds of accepting a competitive position (OR 7.82, p = 0.048; 95 % CI 1.02, 59.4) than the controls. Post-test self-confidence and prior paid employment were not related to accepting a competitive position in this analysis (both p > 0.10).

Table 3.

Odds ratio of VR-JIT as a predictor of attaining a competitive

Odds ratio (95 % C.I.)
Model
Post-training self-confidence 0.91 (0.82–1.02)
Prior paid employment 0.64 (0.13–3.09)
VR-JIT training (yes or no) 7.82 (1.03–59.4)*
Nagelkerke R2 0.21
Open in a new tab
*

p < 0.05

Exploratory Correlations

Among VR-JIT trainees, the observed improvement between pre-test and post-test job interview role-plays during the efficacy study was associated with completing more interviews for a competitive position (r = 0.55, p = 0.02). Advanced social cognition was associated with accepting a competitive position at a trend level (r = 0.37, p = 0.09). No other pairwise correlations among vocational outcomes, neurocognition, ASD symptoms, or VR-JIT processes obtained significance (all p > 0.10).

Discussion

Prior research assessing the efficacy of VR-JIT in a lab-based setting suggested that interviewing skills may improve after simulating job interviews with a virtual character, but the study was limited by a lack of data on whether vocational outcomes were attained after training was completed (Smith et al. 2014b). Thus, we revisited the participants after 6 months to evaluate whether they self-reported having obtained competitive positions after completing an interview. Our findings suggest that VR-JIT trainees, as compared to controls, had 7.82 times greater odds of accepting an offer for a competitive position after controlling for differences in self-confidence and prior paid employment. This finding is consistent with recent evidence that VR-JIT training is associated with better vocational outcomes (Smith et al., in press-b).

There are several directions to continue the evaluation of VR-JIT among adults with ASD. For example, future studies could evaluate whether VR-JIT enhances vocational outcomes (e.g., increased employment rate and reduced time-to-employment) for adults with ASD receiving evidence-based employment services such as Supported Employment or Project Search (Wehman et al. 2012, 2013). Based on the estimated 50,000 adolescents with ASD who will transition out of high school each year (Shattuck et al. 2012a, b), VR-JIT could be modified in an effort to focus the training on the needs of youth with ASD facing this transition (Wehman et al. 2014a). Moreover, VR-JIT requires a 6th grade reading level, which could be modified for individuals with a lower reading level.

Study limitations included having a small sample, which limited statistical power to fully evaluate the effects of VR-JIT and potential mechanisms for these effects. Thus, a community-based effectiveness trial with a larger sample is needed to definitively conclude whether VR-JIT will be helpful to young adults with ASD. A larger sample could be more easily recruited given that VR-JIT uses online learning tutorials and can be widely disseminated via the internet. Of note, we observed a significant odds ratio in spite of limited power, which suggest potentially robust effects of VR-JIT training predicting the acceptance of an offer for a competitive position. We did not verify the study outcomes via employers or volunteer supervisors. Thus, future research could validate the findings by obtaining informant reports from supervisors who may provide objective and more detailed reports of employment or volunteer work. Future studies could also examine potential mechanisms that contribute to the effectiveness of this intervention.

Strengths of the study included capitalizing on previously collected data from randomized samples. Following up with those individuals randomized to VR-JIT or controls allowed us to determine real world effects of the intervention. Moreover, these groups appeared to be similarly motivated to seek competitive positions as the number of completed interviews and weeks searching for a position did not differ between-groups. The analysis of baseline characteristics at 6-month follow-up revealed that groups did not differ with respect to ASD symptoms and other background characteristics. Also, the logistic regression controlled for factors known to influence vocational outcomes (i.e., self-confidence, prior vocational training) (Hall et al. 2011; Tay et al. 2006; Wehman et al. 2014b). Lastly, we evaluated VR-JIT among adults with ASD actively looking for work, which is the target group most likely to take advantage of the intervention. To conclude, vocational training in the community is a critical issue within the ASD community and this study provides initial evidence that VR-JIT may be a helpful intervention for young adults with high-functioning ASD searching for a job or competitive volunteer work.

Acknowledgments

Support for this work was provided by the Department of Psychiatry and Behavioral. The authors acknowledge research staff at Northwestern University’s Clinical Research Program for data collection and our participants for volunteering their time. Dr. Olsen and Laura Boteler-Humm are employed by and own shares in SIMmersion LLC. They contributed to the manuscript, but were not involved in analyzing the data. Dr. Bell was a paid consultant by SIMmersion LLC to assist with the development of VRJIT. Dr. Bell and his family do not have a financial stake in the company.

Footnotes

The remaining authors report no conflicts of interest.

Ethical standard All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

References

  1. Autism Speaks. Employment think tank report (Internet) [Accessed 31 July 2014];2012 http://www.autismspeaks.org/sites/default/files/as_think_tank_exec_summary_web1.pdf. [Google Scholar]
  2. Bell M, Bryson G, Lysaker P. Positive and negative affect recognition in schizophrenia: A comparison with substance abuse and normal control subjects. Psychiatry Research. 1997;73:73–82. doi: 10.1016/s0165-1781(97)00111-x. [DOI] [PubMed] [Google Scholar]
  3. Constantino JN, Gruber CP. Social responsiveness scale, Second Edition (SRS-2) 2nd. Los Angeles, CA: Western Psychological Services; 2012. [Google Scholar]
  4. Gerhardt PF, Lainer I. Addressing the needs of adolescents and adults with autism: A crisis on the horizon. Journal of Contemporary Psychotherapy. 2011;41:37–45. [Google Scholar]
  5. Hall NC, Gradt-Jackson SE, Goetz T, Musu-Gillette LE. Attributional retraining, self-esteem, and the job interview: Benefits and risks for college student employment. Journal of Experimental Education. 2011;79:318–339. [Google Scholar]
  6. Hendricks D. Employment and adults with autism spectrum disorders: Challenges and strategies for success. Journal of Vocational Rehabilitation. 2010;32:125–134. [Google Scholar]
  7. Henninger NA, Taylor JL. Outcomes in adults with autism spectrum disorders: A historical perspective. Autism: The International Journal of Research and Practice. 2013;17:103–116. doi: 10.1177/1362361312441266. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Higgins KK, Koch LC, Boughfman EM, Vierstra C. School-to-work transition and Asperger Syndrome. Work. 2008;31:291–298. [PubMed] [Google Scholar]
  9. Liptak GS, Kennedy JA, Dosa NP. Social participation in a nationally representative sample of older youth and young adults with autism. Jouranl of Developmental and Behavioral Pediatrics. 2011;32:1–7. doi: 10.1097/DBP.0b013e31820b49fc. [DOI] [PubMed] [Google Scholar]
  10. Morgan L, Leatzow A, Clark S, Siller M. Interview skills for adults with autism spectrum disorder: A pilot randomized controlled trial. Journal of Autism and Developmental Disorders. 2014 doi: 10.1007/s10803-014-2100-3. [DOI] [PubMed] [Google Scholar]
  11. Randolph C, Tierney MC, Mohr E, Chase TN. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): preliminary clinical validity. Journal of Clinical and Experimental Neuropsychology. 1998;20:310–319. doi: 10.1076/jcen.20.3.310.823. [DOI] [PubMed] [Google Scholar]
  12. Shattuck PT, Narendorf SC, Cooper B, Sterzing PR, Wagner M, Taylor JL. Postsecondary education and employment among youth with an autism spectrum disorder. Pediatrics. 2012a;129:1042–1049. doi: 10.1542/peds.2011-2864. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Shattuck PT, Roux AM, Hudson LE, Taylor JL, Maenner MJ, Trani JF. Services for adults with an autism spectrum disorder. Canadian Journal of Psychiatry. 2012b;57:284–291. doi: 10.1177/070674371205700503. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Smith MJ, et al. Virtual reality job interview training for individuals with psychiatric disabilities. The Journal of Nervous and Mental Disease. 2014a;202(9):659–667. doi: 10.1097/NMD.0000000000000187. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Smith MJ, et al. Virtual reality job interview training in adults with autism spectrum disorder. Journal of Autism and Developmental Disorders. 2014b;44(10):2450–2463. doi: 10.1007/s10803-014-2113-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Smith MJ, et al. Performance-based empathy mediates the influence of working memory on social competence in schizophrenia. Schizophrenia Bulletin. 2014c;40:824–834. doi: 10.1093/schbul/sbt084. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Smith MJ, et al. Virtual reality job interview training for veterans with posttraumatic stress disorder. Journal of Vocational Rehabilitation. doi: 10.3233/JVR-150748. (in press-a) [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Smith MJ, et al. Job offers among individuals with severe mental illness after virtual reality job interview training. Psychiatric Services. doi: 10.1176/appi.ps.201400504. (in press-b) [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Tay C, Ang S, Van Dyne L. Personality, biographical characteristics, and job interview success: A longitudinal study of the mediating effects of interviewing self-efficacy and the moderating effects of internal locus of causality. Journal of Applied Psychology. 2006;91:446–454. doi: 10.1037/0021-9010.91.2.446. [DOI] [PubMed] [Google Scholar]
  20. Taylor JL. The transition out of high school and into adulthood for individuals with autism and for their families. International Review of Research in Mental Retardation. 2009;38:1–32. [Google Scholar]
  21. Volkmar FR, State M, Klin A. Autism and autism spectrum disorders: Diagnostic issues for the coming decade. Journal of Child Psychology and Psychiatry. 2009;50:108–115. doi: 10.1111/j.1469-7610.2008.02010.x. [DOI] [PubMed] [Google Scholar]
  22. Wehman P, Schall C, Carr S, Targett P, West M, Cifu G. Transition from school to adulthood for youth with ASD: What we know and what we need to know. Journal of Disability Policy Studies. 2014a;25:30–40. [Google Scholar]
  23. Wehman P, Schall CM, McDonough J, Molinelli A, Riehle E, Ham W. Project search for youth with autism spectrum disorders: Increasing competitive employment on transition from high school. Journal of Positive Behavior Intervention. 2013;15:144–155. [Google Scholar]
  24. Wehman P, et al. Supported employment for young adults with autism spectrum disorder: Preliminary data. Research and Practice for Persons with Severe Disabilities. 2012;37:160–169. [Google Scholar]
  25. Wehman PH, et al. Competitive employment for youth with autism spectrum disorders: Early results from a randomized clinical trial. Journal of Autism and Developmental Disorders. 2014b;44:487–500. doi: 10.1007/s10803-013-1892-x. [DOI] [PubMed] [Google Scholar]

RESOURCES