Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Oct 25.
Published in final edited form as: J Vocat Rehabil. 2016 May 7;44(3):323–332. doi: 10.3233/JVR-160802

Virtual Reality Job Interview Training and 6-Month Employment Outcomes for Individuals with Substance Use Disorders Seeking Employment

Matthew J Smith 1,2,*, Morris D Bell 3, Michael A Wright 1, Laura Boteler Humm 4, Dale Olsen 4, Michael F Fleming 1,5
PMCID: PMC6814393  NIHMSID: NIHMS1050899  PMID: 31656389

Abstract

BACKGROUND:

Individuals with substance use disorders (SUDs) have low employment rates and job interviewing is a critical barrier to employment for them. Virtual reality training is efficacious at improving interview skills and vocational outcomes for several clinical populations.

OBJECTIVE:

This study evaluated the acceptability and efficacy of virtual reality job interview training (VR-JIT) at improving interview skills and vocational outcomes among individuals with SUDs via a small randomized controlled trial (n=14 VR-JIT trainees, n=11 treatment-as-usual (TAU) controls).

METHODS:

Trainees completed up to 10 hours of virtual interviews, while controls received services as usual. Primary outcome measures included two pre-test and two post-test video-recorded role-play interviews and vocational outcomes at six-month follow-up.

RESULTS:

Trainees reported that the intervention was easy-to-use and helped prepared them for future interviews. While co-varying for pre-test role-play performance, trainees had higher post-test role-play scores than controls at the trend level (p<0.10). At 6-month follow-up, trainees were more likely than controls to attain a competitive position (78.6% vs. 44.4%, p<0.05, respectively). Trainees had greater odds of attaining a competitive position by 6 month follow-up compared to controls (OR: 5.67, p<0.05). VR-JIT participation was associated with fewer weeks searching for a position (r= −0.36, p<0.05).

CONCLUSIONS:

There is preliminary evidence that VR-JIT is acceptable to trainees. Moreover, VR-JIT led to better vocational outcomes with trainees having greater odds of attaining a competitive position by 6-month follow-up. Future studies could evaluate the effectiveness of VR-JIT within community-based services.

Keywords: substance use disorders, virtual reality training, job interview skills, vocational outcomes

1. Introduction

Individuals diagnosed with an alcohol or other substance use disorders (SUDs) have low levels of competitive employment (15–30%) (McCoy, Comerford, & Metsch, 2007; Platt, 1995; Richardson, Wood, Li, & Kerr, 2010; Sigurdsson, DeFulio, Long, & Silverman, 2011). In turn, research has evaluated barriers to employment for people with SUDs in an effort to identify targets for intervention development (Richardson et al., 2010; Richardson, Wood, Montaner, & Kerr, 2012; Sigurdsson, Ring, O’Reilly, & Silverman, 2012). Findings from this research suggest that they face barriers to employment that include the lack of on-the-job ‘hard’ skills (e.g., computer skills) (Sigurdsson et al., 2012) and impairments in ‘soft’ social skills that impact on-the-job behavior (e.g., interacting with customers and coworkers) as well as pre-job behavior (e.g., job interviewing) (Ginexi, 2003; Lidz, Sorrentino, Robison, & Bunce, 2004). Some addiction treatment programs target both hard and soft skills in an effort to help these individuals attain volunteer positions or competitive jobs as both of these outcomes are critical to recovery (Pagano et al., 2009; Room, 1998; White, 2009; Zemore, Kaskutas, & Ammon, 2004) and contribute to better quality of life (Donovan, Mattson, Cisler, Longabaugh, & Zweben, 2005; Foster, Peters, & Marshall, 2000; Laudet, 2011; Laudet, Becker, & White, 2009).

Individuals with SUDs face similar barriers to employment (e.g., lack of skills, fear of benefit loss) as individuals with severe mental illness (SMI) (Cook, 2006). However, few studies have evaluated whether individuals with SUDs may be responsive to evidence-based supported employment (SE) that helps individuals with SMI overcome those barriers (Drake & Bond, 2011). This trend may be changing as some studies have begun to evaluate the effectiveness of SE in populations with co-occurring SMI and SUDs (Frounfelker, Wilkniss, Bond, Devitt, & Drake, 2011; Rosenheck & Mares, 2007). Despite a paucity of research examining job interview skills among individuals with SUDs, it would be reasonable to suggest that this clinical population recognizes the job interview as a gateway to competitive volunteer work or employment as they face similar barriers to employment as individuals with SMI (and has limited access to standardized vocational services).

Recently, a series of randomized controlled trials (RCTs) evaluated the efficacy of Virtual Reality Job Interview Training (VR-JIT), an intervention that helps trainees prepare for job interviews. VR-JIT has demonstrated acceptability and efficacy at improving interview skills in multiple clinical cohorts, including: individuals with mood disorders, veterans with post-traumatic stress disorder (PTSD), individuals with schizophrenia, and young adults with autism spectrum disorder (Smith et al., 2015a; Smith et al., 2015d; Smith et al., 2014a; Smith et al., 2014b). Moreover, VR-JIT trainees were found to have better vocational outcomes (e.g., attain job offers; get offers faster) than their comparison groups; and more completed virtual interview trials were associated with an increased likelihood of receiving a job offer and reduced time searching for jobs (Smith et al., 2015b; Smith et al., 2015c; Smith et al., 2015d).

Based on prior evaluations of VR-JIT, our primary hypothesizes were that 1) trainees would have greater improvement in job interview skills as evaluated through pre-post standardized role-plays and 2) trainees would have better vocational outcomes (i.e., attaining a job or competitive volunteer position) compared to controls at 6-month follow-up. Our secondary hypotheses were that participation in the VR-JIT group (and amount of VR-JIT) would be associated with weeks-to-outcome and role-play performance, and that trainees would find VR-JIT easy-to-use and helpful. We generated these directional hypotheses based on our prior work (Smith et al., 2015b; Smith et al., 2015c; Smith et al., 2015d).

2. Methods

2.1. Participants

Participants included 25 individuals with a primary Alcohol or other Substance Use Disorder recruited through Northwestern University. Inclusion criteria included: 18–65 years old, minimum of a 6th grade reading level using the Wide Range Achievement Test-IV (WRAT-IV) (Wilkinson & Robertson, 2006), willingness to be video-recorded, unemployed or underemployed, actively seeking employment, lifetime history of abusing alcohol or drugs, and actively receiving outpatient treatment verified by their case manager (e.g., substance use counseling, individuals psychotherapy, vocational rehabilitation). The study exclusion criteria included: having a medical illness that significantly comprised cognition (e.g., traumatic brain injury), uncorrected vision or hearing problem. Northwestern University’s Institutional Review Board approved the study protocol and all participants provided informed consent. Once enrolled, participants were randomized into the training (n=14) or treatment-as-usual control (n=11) groups. Data from 2 trainees who met inclusion criteria and had completed prior studies were included in the present analyses to optimize statistical power (Smith et al., 2015a; Smith et al., 2014b). Participants were re-contacted after 6 months and asked to complete a follow-up survey. Of the original 25 participants, 23 (92%) completed the follow-up survey and 2 (8%) were lost to contact.

2.2. Intervention

Virtual Reality Job Interview Training (VR-JIT) is a computer-based intervention developed by SIMmersion LLC (http://www.simmersion.com) to enhance interviewing skills for individuals with a range of disabilities. Trainees review interviewing didactics and repeatedly practice job interviews with Molly Porter (a virtual human resources representative) using speech recognition. The training was developed based on 8 learning goals: 1) sounding like a hard worker, 2) sounding easy to work with, 3) behaving professionally, 4) negotiation skills (asking for Thursdays off), 5) sharing things in a positive way, 6) sounding honest, 7) sounding interested in the position and 8) establishing overall rapport with the interviewer. Please visit http://www.jobinterviewtraining.net to view images of Molly and the VR-JIT interface.

VR-JIT was designed to improve interview skills using behavioral learning principles(Cooper, 1982; Cooper, Heron, & Heward, 2007) and Issenberg et al’s principles for designing effective simulations (Issenberg, 2006). These principals are noted help develop sustainable changes in behavior (Roelfsema, van Ooyen, & Watanabe, 2010; Vinogradov, Fisher, & de Villers-Sidani, 2012). Virtual job interview performances were scored on a scale of 0–100 and ranged in difficulty from easy to medium to hard. Participants were required to score 90 or better within 3 trials on a given difficulty level before advancing to the next level. If a score of 90 was not attained then participants were automatically advanced after 5 trials. Upon completion, trainees received feedback on each response that they could review and use as a learning tool. See Smith et al. (2014b) for additional details on design and delivery (e.g., fidelity training) of VR-JIT.

2.3. Study Procedures

Pre-test measures included: demographic, clinical, cognitive, and vocational assessments and two standardized role-plays. Participants were randomized following completion of the pre-test assessments. Trainees completed up to 10 hours of VR-JIT (~20 trials) over the span of 5 visits (within 5–10 business days). Controls received services-as-usual during this same time frame. After 10 days, both groups completed two post-test standardized role-plays, and trainees completed the Treatment Experience Questionnaire (TEQ).

Approximately 6 months after completing the above efficacy trial, research staff contacted participants to complete a brief follow-up survey over the phone or via email. Two controls were unreachable by phone, mail, and email, and were lost to contact. Overall, 14 VR-JIT and 9 controls completed follow-up.

2.4. Study Measures

2.4.1. Participant Characteristics

We assessed demographic characteristics and vocational history via self-report and a Bachelor’s or Ph.D.-level research staff assessed addiction severity and mental health using the Addiction Severity Index (ASI) (McLellan, Luborsky, Woody, & O’Brien, 1980). The ASI measures the problem severity for the 30 days prior to the interview in several domains of functioning: medical status, family/social status, employment and support, psychiatric status (e.g., mood), legal, drug use, and alcohol use. Alcohol and drug use data include: main substance used, months abstinent, days of addiction outpatient treatment, days of use over past 30 days, and years of use. The history of addiction and treatment were validated for the two participants from prior studies using the Mini-International Neuropsychiatric Interview (MINI) to determine DSM-IV Axis I diagnoses (Sheehan et al., 1998).

2.4.2. Cognition

We measured neurocognition using the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; (Randolph, Tierney, Mohr, & Chase, 1998). The RBNAS total score reflects performance across immediate memory, visuospatial capacity, language, attention, and delayed memory.

2.4.3. VR-JIT Acceptability

Trainee attendance and the total number of minutes (600 minutes possible) across the five sessions that they engaged in virtual interviews were recorded. Trainees completed the TEQ to assess if they thought VR-JIT was easy to use, enjoyable, helpful, instilled confidence, and prepared them for interviews (Bell & Weinstein, 2011).

2.4.4. VR-JIT Efficacy

The scoring criteria for the job interview role-plays (~20 minutes each) included nine communication skills that are critical for performing a successful interview (Huffcutt, 2011); 1) conveying oneself as a hard worker, 2) sounding easy to work with, 3) conveying that one behaves professionally, 4) negotiation skills (requesting Thursdays off), 5) sharing things in a positive way, 6) sounding honest, 7) sounding interested in the position, 8) comfort level, and 9) establishing rapport.

Participants filled out a job application and completed two pre-test and two post-test role-plays. Each participant chose four job scenarios for the role-plays (e.g., Maintenance, Librarian). Participants were instructed to negotiate for a work schedule where they could take Thursdays off for personal reasons. Standardized role-play actors (SRAs) were trained to pose as human resources representatives. They interviewed participants by asking 13 standardized questions and 3–4 random questions (70+ questions available). Role-plays were video-recorded for scoring purposes.

Videos were rated in a random order by one blinded rater who had more than 10 years of experience in human resources and who had served as a rater for our prior studies (Smith et al., 2015a; Smith et al., 2014a; Smith et al., 2014b). The rater trained with 10 practice videos before independently rating the study videos. See Smith et al. (2014b, 2015a) for additional information about these methods. Total scores for each of the two baseline and follow-up role-plays were computed across nine domains (range of 1–5 with higher scores reflecting better performance), and averaged to compute a single score. Despite SRA prompting, 25% of the role plays did not include a negotiation for Thursdays off. An item-level imputation for this score replaced missing ratings (Myers, 2000). No other ratings were missing.

2.4.5. VR-JIT Process Measures

We recorded trainees’ VR-JIT performance scores, number of completed trials, and time spent engaged in virtual interviews. The software scored each virtual interview from 0–100 using an algorithm that targeted their responses in eight domains: negotiation skills (asking for Thursdays off), conveying that you’re a hard worker, sounding easy to work with, sharing things in a positive way, sounding honest, sounding interested in the position, behaving professionally, and establishing interviewer rapport.

2.4.6. Six-month follow-up measures

The follow-up survey asked participants to reflect on the past 6 months and report 1) total number of weeks they spent searching for a job or volunteer position, 2) number of job or volunteer interviews completed, and 3) number of job or volunteer offers accepted. The 6-month survey also assessed whether trainees believed that VR-JIT prepared them for real interviews, helped them attain employment, and revisiting the training would help them prepare for future interviews. The 5 items were rated on a 5-point Likert scale (1=strongly disagree to 5=strongly agree).

2.5. Data analysis

2.5.1. Primary outcomes for efficacy study

Between-group differences for demographics, vocational history, cognition, and clinical history were assessed with a Mann-Whitney independent samples test or chi-square analysis. We analyzed VR-JIT acceptability using descriptive statistics for session attendance, the number of minutes engaged with VR-JIT, and TEQ responses. We evaluated whether role-play performance for trainees significantly differed at post-test as compared to controls when co-varying for pre-test via an analysis of covariance (ANCOVA).

We evaluated whether trainees improved their VR-JIT performance scores across trials as a process measure by computing linear regression ‘learning’ slopes for each trainee based on the regression of their performance scores on the log of trial number. We plotted the group-level performance average for each successive VR-JIT trial and generated the R-Square from the regression of average performance on the log of trial number.

2.5.2. Primary outcomes at six month follow-up

We conducted a logistic regression with attaining a job or competitive volunteer position (1=yes, 0=no) as the dependent variable to evaluate whether or not trainees had higher odds of this outcome than controls. Neurocognition and the number of months since prior employment were included as covariates based on their a priori relationship to vocational outcomes in populations with severe mental illness (Burke-Miller et al., 2006; Catty et al., 2008; Gold, Goldberg, McNary, Dixon, & Lehman, 2002). Odds ratios (OR) were generated and presented with 95% Confidence Intervals. Nagelkerke R-Squared provided the model’s proportion of explained variance.

2.5.3. Secondary outcomes

We conducted point serial correlations to evaluate whether group status (training vs. control) was associated with fewer weeks-to-outcome at 6-month follow-up. We conducted Pearson correlations to evaluate whether VR-JIT process measures (i.e., number of completed trials, learning slope) were correlated with role-play performance and weeks-to-outcome.

3. Results

3.1. Pre-test between-group characteristics

In Table 1, trainees and controls did not differ with respect to age, race, parental education, vocational history, cognition, days of outpatient treatment (past month), months of current abstinence, and years of cocaine and heroin use (all p>0.10, df=1 for nominal variables). Despite random assignment, the control group had a longer history (in years) of using cannabis (p<0.01) and alcohol (trend level, p=0.07).

Table 1.

Baseline participant characteristics

Control Group (n=11) Training Group (n=14) p-value
Demographics
 Mean age (SD) 52.2 (6.1) 51.9 (6.0) 0.65
 Gender (% male) 54.5% 64.3% 0.62
 Parental education, mean years (SD) 11.1 (4.1) 12.4 (2.3) 0.78
 Race
  % Caucasian 18.2% 28.6%
  % African-American 72.7% 71.4% 0.46
  % Latino 9.1% 0.0%
Vocational history
 Months since prior employment, mean (SD) 53.8 (59.1) 75.5 (86.4) 0.77
 Prior full-time employment (%) 100% 100% --
 Prior participation in vocational training program 45.5% 21.4% 0.20
Neurocognitive function, mean (SD) 82.9 (11.9) 85.7 (13.7) 0.89
Substance Use History
 Days of outpatient treatment (past 30 days)a 14.1 (6.2) 14.7 (9.7) 0.98
 Months of current abstinencea 43.0 (47.9) 87.8 (97.5) 0.43
 Primary substance of abusea
  Alcohol 18.2% 28.6%
  Polydrugb (and alcohol) 72.7% 35.7%
  Polydrugb (no alcohol) 0.0% 28.6% 0.13
  Cocaine 7.1% 7.1%
  Heroin 0.0% 0.0%
Years of Usea
  Alcohol, mean (SD) 27.09 (12.8) 16.08 (12.6) 0.07
  Cannabis, mean (SD) 17.00 (10.1) 10.11 (3.0) 0.007
  Cocaine, mean (SD) 9.55 (9.6) 8.33 (6.7) 0.93
  Heroin, mean (SD) 7.36 (13.1) 7.42 (7.2) 0.79
Open in a new tab
a

Data missing from two trainees;

b

Heroin, cocaine, and cannabis are the noted drugs used.

3.2. VR-JIT acceptability

In Table 2, trainees attended mean=548.6 (sd=90.7) minutes of VR-JIT and completed mean=17.0 (sd=3.0) trials. Trainees reported that VR-JIT was easy to use, enjoyable, helpful, increased their self-confidence in job-interview skills, and improved their readiness for interviewing.

Table 2.

Mean characteristics of VR-JIT acceptability (SD)

Attendance Measures
 Completed Trials (out of 20) 17.2 (3.0)
 Elapsed Simulation Time (min) 548.6 (90.7)
TEQ Items
 Ease of use 5.9 (1.3)
 Enjoyable 6.1 (1.1)
 Helpful 6.4 (1.2)
 Instilled confidence 6.2 (1.2)
 Prepared for interviews 6.4 (0.8)
Open in a new tab

Note. Scale for TEQ, 1=Extremely Unhelpful to 7=Extremely Helpful;

Abbreviations: VR-JIT, virtual reality job interview training;

TEQ, training experience questionnaire.

3.3. VR-JIT process measures

The process measures indicated that VR-JIT performance scores appeared to improve linearly across the number of completed trials (Figure 1). The slope (mean=2.6, sd=2.0) suggests that performance improves 2.6 points for every 1 point increase in the natural log of the trial number (R-Squared= 0.64).

Figure 1.

Figure 1.

Open in a new tab

VR-JIT Learning Curve in Individuals with Substance Use Disorders. This figure plots the average score for each successive VR-JIT simulated interview trial. Trials 1–3 at easy, trials 4–6 at medium, and trials 7–17 at hard. Model fit, R2=.64

3.4. Primary Outcomes for Efficacy Study

In Figure 2, ANCOVA revealed a trend-level group effect that VR-JIT trainees, as compared to controls, had higher post-test role-play performance scores (M=38.2, SD=1.9 vs. M=37.2, SD=1.9, respectively) when covarying for pre-test scores (F(1,22)=1.8, p=0.097).

Figure 2.

Figure 2.

Open in a new tab

Differences in post-test role-play Interview performance for each group while covarying for pre-test role-play performance.

3.5. Primary outcomes at 6-month follow-up

In Table 3, we report that a similar proportion of controls and trainees completed interviews for a job or volunteer position (p>0.10, df=1). More trainees attained jobs or volunteer positions than controls (p=0.047, df=1). Trainees, as compared to controls, completed fewer interviews (p=0.04) and looked for a position for fewer weeks (trend, p=0.07).

Table 3.

Six month follow-up between-group differences

N Control Group (n=9) N Training Group (n=14) p-value
Mean total weeks looking for a job (SD) 16.7 (9.6) 9.3 (10.1) 0.07
Mean total job or volunteer interviews completed (SD) 6.0 (4.4) 2.6 (1.9) 0.04
 % who completed job or volunteer interviews 9 100% 13 92.9% 0.67
 % who attained job or volunteer positiona 4 44.4% 11 78.6% 0.047
  % who attained a job 3 33.3% 8 57.1% 0.13
  % who attained a volunteer position 1 11.1% 8 30.8% 0.14
Open in a new tab
a

Five participants attained a job and volunteer position.

In Table 4, we report the odds of attaining a position (job or volunteer) were 5.67 times higher for trainees compared to controls (OR=5.67, df=1, p=0.043; 95% CI=1.07, 30.04). Neurocognition and months since prior employment were non-significant predictors (p>0.10). Overall, the model explained 18.7% of the variance in position attainment (Nagelkerke R-Squared=0.187).

Table 4.

VR-JIT as a predictor of attaining a competitive position

Step 1
OR (C.I. 95%)		 Step 2
OR (C.I. 95%)
Step 1a
 Neurocognition 1.00 (0.94–1.05) 0.98 (0.92–1.05)
 Months since prior employment 1.00 (0.99–1.01) 1.00 (0.99–1.01)
Step 2b
 VR-JIT (yes or no) -- 5.67 (1.07–30.04)*
Nagelkerke R2 0.01 0.19
Open in a new tab
a

Step 1 Omnibus Test of Model Coefficients, Chi-Square=0.09, df=2, p=0.95

b

Step 2 Omnibus Test of Model Coefficients, Chi-Square=3.26 df=3, p=0.035

Abbreviations: VR-JIT, virtual reality job interview training;

3.6. Secondary outcomes

Participation in VR-JIT (i.e., group status) was correlated with fewer weeks searching for a job (r=−0.36, p<0.05). A larger VR-JIT performance slope correlated with greater improvement in role-play performance (r=0.57, p<0.05). The completion of a greater number of VR-JIT trials correlated with improved role-play performance at a trend level (r=0.40, p=0.08). Correlations between number of trials and learning slope with weeks-to-outcome were non-significant (both p>0.10).

At 6-month follow-up, all trainees agreed or strongly agreed that VR-JIT prepared them for real interviews and that the training was helpful to them. Over 85% of trainees agreed or strongly agreed that they would use VR-JIT again to enhance their skills.

4. Discussion

We evaluated both the acceptability and efficacy of VR-JIT in a small RCT of individuals with substance use disorders who self-reported that they were actively looking for employment. The trainees found VR-JIT to be acceptable based on their reports on its ease-of-use, it being enjoyable to use, and its helpfulness in preparing for future interviews. Moreover, VR-JIT appears to demonstrate efficacy at improving role-play performance and trainees learned from the intervention as their virtual interview scores increased across greater levels of difficulty. The 6-month follow-up data suggests trainees had greater odds of attaining a job or competitive volunteer position. Of note, the analyses controlled for known predictors of vocational outcomes (i.e., cognition, time since prior employment) (Burke-Miller et al., 2006; Catty et al., 2008; Gold et al., 2002). Also, using VR-JIT was associated with a shorter duration of searching for a job or volunteer position, and trainees reported that VR-JIT prepared them for real-life interviews.

Although trainees improved their interviewing skills, this effect was at the trend level. The general direction of this finding was consistent with our recent evaluation of VR-JIT in several other clinical populations (Smith et al., 2015a; Smith et al., 2014a; Smith et al., 2014b). Moreover, we observed that trainees had greater odds of attaining a position and spent fewer weeks searching for positions as compared to controls. These findings also replicate our 6-month follow-up data among veterans with PTSD, and individuals with either a mood disorder or schizophrenia, and young adults with autism spectrum disorders (Smith et al., 2015b; Smith et al., 2015c; Smith et al., 2015d). Moreover, improving access to employment and competitive volunteer work is notable as these outcomes are associated with a higher quality of life (Donovan et al., 2005; Foster et al., 2000; Laudet, 2011; Laudet et al., 2009).

The current findings suggest that there are several directions for future research. First, the findings need to be replicated in a larger sample to validate that the training is efficacious for adults with SUDs. Second, most participants have limited access to standardized vocational services. Thus, VR-JIT could be evaluated as a complement to vocational services that are currently available. Third, the current study focused on midlife adults, while future studies could evaluate whether VR-JIT is effective for younger individuals in recovery from SUDs. Lastly, most participants were last employed 5–10 years previously. Perhaps future studies could evaluate whether VR-JIT may be more helpful to individuals who were more recently employed.

Based on the pilot nature of the current study, we would like to offer a few implications as hypotheses to be tested in dissemination and implementation studies. For instance, VR-JIT is likely to be scalable to both small- and large-scale vocational rehabilitation programs due to its ability to be self-sustaining. More specifically, the use of a training manual and the self-directed nature of VR-JIT suggests that vocational counselors may not be needed to guide their clients through job interview training. As such, vocational counselors could shift their efforts from conducting job interview training to soliciting additional community-based jobs for their caseload, train clients in employable skills, or conduct other job development duties. Thus, future studies can evaluate the scalability of VR-JIT, and whether its implementation has downstream effects on vocational rehabilitation programming. Also, future research could conduct a budget impact or cost effectiveness analysis to evaluate the impact of VR-JIT on programming budgets.

There were some limitations that much be considered when interpreting the findings. The extent of services received by the treatment-as-usual group (e.g., contact hours, skills taught, performance feedback) were not monitored due to the limited resources of this pilot study. Future studies would need to carefully monitor these important variables. The study has limited statistical power given the sample size. The 6-month outcome data does not include types of jobs attained or pay received, and the duration of the position. All study participants were actively seeking work or a volunteer position. Thus, the results do not generalize to individuals not actively searching for these engagements. However, job-seekers are the individuals who are likely to use the training so the findings are generalizable to them. The participants were paid for completing the study, which may have biased the results.

6. Conclusions

VR-JIT may be a helpful tool for individuals with SUDs as training was associated increasing the odds that trainees obtained a job or competitive volunteer work and a trend towards improved interview skills. VR-JIT also helped trainees attain these positions faster as compared to controls. In the future, a definitive study is still needed to evaluate VR-JIT effectiveness within a community setting. Based on the ability to download and use VR-JIT in a rapid fashion, this intervention could be widely disseminated to clinics and treatment centers that have limited access to vocational services.

Acknowledgements

Support for this work was provided by the Department of Psychiatry and Behavioral Sciences. The authors would like to acknowledge Dr. Zoran Martinovich for advising on statistical issues, the research staff at Northwestern University’s Clinical Research Program for data collection, and our participants for volunteering their time.

Funding Source

The development of VR-JIT was supported by a grant from the National Institute of Mental Health awarded to Dr. Dale Olsen (R44 MH080496). The evaluation of VR-JIT in the current cohort of participants was supported by the Department of Psychiatry and Behavioral Sciences at Northwestern University.

Footnotes

Conflict of Interest

Dr. Olsen and Laura Boteler-Humm are employed by and own shares in SIMmersion LLC. They contributed to the manuscript, but were not involved in analyzing the data. Dr. Bell was a paid consultant by SIMmersion LLC to assist with the development of VR-JIT. Dr. Bell and his family do not have a financial stake in the company. The remaining authors report no conflicts of interest.

References

  1. Bell MD, & Weinstein A (2011). Simulated job interview skill training for people with psychiatric disability: feasibility and tolerability of virtual reality training. Schizophrenia Bulletin, 37 Suppl 2, S91–97. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Burke-Miller JK, Cook JA, Grey DD, Razzano LA, Blyler CR, Leff HS, Gold PB, Goldberg RW, Mueser KT, Cook WL, Hoppe SK, Stewart M, Blankertz L, Dudek K, Taylor AL, & Carey MA (2006). Demographic characteristics and employment among people with severe mental illness in a multisite study. Community Ment Health J, 42, 143–159. [DOI] [PubMed] [Google Scholar]
  3. Catty J, Lissouba P, White S, Becker T, Drake RE, Fioritti A, Knapp M, Lauber C, Rossler W, Tomov T, van Busschbach J, Wiersma D, & Burns T (2008). Predictors of employment for people with severe mental illness: results of an international six-centre randomised controlled trial. Br J Psychiatry, 192, 224–231. [DOI] [PubMed] [Google Scholar]
  4. Cook JA (2006). Employment barriers for persons with psychiatric disabilities: update of a report for the President’s Commission. Psychiatr Serv, 57, 1391–1405. [DOI] [PubMed] [Google Scholar]
  5. Cooper JO (1982). Applied behavior analysis in education. Theory Into Practice, 21, 114–118. [Google Scholar]
  6. Cooper JO, Heron TE, & Heward WL (2007). Applied Behavioral Analysis. London: Pearson. [Google Scholar]
  7. Donovan D, Mattson ME, Cisler RA, Longabaugh R, & Zweben A (2005). Quality of life as an outcome measure in alcoholism treatment research. J Stud Alcohol Suppl, 119–139; discussion 192–113. [DOI] [PubMed] [Google Scholar]
  8. Drake RE, & Bond GR (2011). IPS Supported Employment: A 20 year Update. American Journal of Psychiatric Rehabilitation, 14, 155–164. [Google Scholar]
  9. Foster JH, Peters TJ, & Marshall EJ (2000). Quality of life measures and outcome in alcohol-dependent men and women. Alcohol, 22, 45–52. [DOI] [PubMed] [Google Scholar]
  10. Frounfelker RL, Wilkniss SM, Bond GR, Devitt TS, & Drake RE (2011). Enrollment in supported employment services for clients with a co-occurring disorder. Psychiatr Serv, 62, 545–547. [DOI] [PubMed] [Google Scholar]
  11. Ginexi EMF, M.A.; Scott CK (2003). Transitions from treatment to work: Employment patters following publicly funded substance abuse treatment. Journal of Drug Issues, 33, 497–518. [Google Scholar]
  12. Gold JM, Goldberg RW, McNary SW, Dixon LB, & Lehman AF (2002). Cognitive correlates of job tenure among patients with severe mental illness. Am J Psychiatry, 159, 1395–1402. [DOI] [PubMed] [Google Scholar]
  13. Huffcutt AI (2011). An empirical review of the employment interview construct literature. International Journal of Selection and Assessment, 19, 62–81. [Google Scholar]
  14. Issenberg SB (2006). The scope of simulation-based healthcare education. Simul Healthc, 1, 203–208. [DOI] [PubMed] [Google Scholar]
  15. Laudet AB (2011). The case for considering quality of life in addiction research and clinical practice. Addict Sci Clin Pract, 6, 44–55. [PMC free article] [PubMed] [Google Scholar]
  16. Laudet AB, Becker JB, & White WL (2009). Don’t wanna go through that madness no more: quality of life satisfaction as predictor of sustained remission from illicit drug misuse. Subst Use Misuse, 44, 227–252. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Lidz V, Sorrentino DM, Robison L, & Bunce S (2004). Learning from disappointing outcomes: an evaluation of prevocational interventions for methadone maintenance patients. Subst Use Misuse, 39, 2287–2308. [DOI] [PubMed] [Google Scholar]
  18. McCoy CB, Comerford M, & Metsch LR (2007). Employment among chronic drug users at baseline and 6-month follow-up. Subst Use Misuse, 42, 1055–1067. [DOI] [PubMed] [Google Scholar]
  19. McLellan AT, Luborsky L, Woody GE, & O’Brien CP (1980). An improved diagnostic evaluation instrument for substance abuse patients. The Addiction Severity Index. J Nerv Ment Dis, 168, 26–33. [DOI] [PubMed] [Google Scholar]
  20. Myers WR (2000). Handling missing data in clinical trials: an overview. Drug Information Journal, 34, 525–533. [Google Scholar]
  21. Pagano ME, Zeltner BB, Jaber J, Post SG, Zywiak WH, & Stout RL (2009). Helping Others and Long-term Sobriety: Who Should I Help to Stay Sober? Alcohol Treat Q, 27, 38–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Platt JJ (1995). Vocational rehabilitation of drug abusers. Psychol Bull, 117, 416–433. [DOI] [PubMed] [Google Scholar]
  23. Randolph C, Tierney MC, Mohr E, & Chase TN (1998). The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): preliminary clinical validity. J Clin Exp Neuropsychol, 20, 310–319. [DOI] [PubMed] [Google Scholar]
  24. Richardson L, Wood E, Li K, & Kerr T (2010). Factors associated with employment among a cohort of injection drug users. Drug Alcohol Rev, 29, 293–300. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Richardson L, Wood E, Montaner J, & Kerr T (2012). Addiction treatment-related employment barriers: the impact of methadone maintenance. J Subst Abuse Treat, 43, 276–284. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Roelfsema PR, van Ooyen A, & Watanabe T (2010). Perceptual learning rules based on reinforcers and attention. Trends Cogn Sci, 14, 64–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Room JA (1998). Work and identity in substance abuse recovery. J Subst Abuse Treat, 15, 65–74. [DOI] [PubMed] [Google Scholar]
  28. Rosenheck RA, & Mares AS (2007). Implementation of supported employment for homeless veterans with psychiatric or addiction disorders: two-year outcomes. Psychiatr Serv, 58, 325–333. [DOI] [PubMed] [Google Scholar]
  29. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, Hergueta T, Baker R, & Dunbar GC (1998). The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry, 59 Suppl 20, 22–33;quiz 34–57. [PubMed] [Google Scholar]
  30. Sigurdsson SO, DeFulio A, Long L, & Silverman K (2011). Propensity to work among chronically unemployed adult drug users. Subst Use Misuse, 46, 599–607. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Sigurdsson SO, Ring BM, O’Reilly K, & Silverman K (2012). Barriers to employment among unemployed drug users: age predicts severity. Am J Drug Alcohol Abuse, 38, 580–587. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Smith MJ, Boteler Humm L, Fleming MF, Jordan N, Wright MA, Ginger EJ, Wright K, Olsen D, & Bell MD (2015a). Virtual Reality Job Interview Training For Veterans with Posttraumatic Stress Disorder. Journal of Vocational Rehabilitation, 42, 271–279. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Smith MJ, Fleming MF, Wright MA, Jordan N, Humm LB, Olsen D, & Bell MD (2015b). Job Offers to Individuals With Severe Mental Illness After Participation in Virtual Reality Job Interview Training. Psychiatr Serv, appips201400504. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Smith MJ, Fleming MF, Wright MA, Losh M, Humm LB, Olsen D, & Bell MD (2015c). Brief Report: Vocational Outcomes for Young Adults with Autism Spectrum Disorders at Six Months After Virtual Reality Job Interview Training. J Autism Dev Disord. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Smith MJ, Fleming MF, Wright MA, Roberts AG, Humm LB, Olsen D, & Bell MD (2015d). Virtual reality job interview training and 6-month employment outcomes for individuals with schizophrenia seeking employment. Schizophr Res, 166, 86–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Smith MJ, Ginger EJ, Wright K, Wright MA, Taylor JL, Humm LB, Olsen DE, Bell MD, & Fleming MF (2014a). Virtual reality job interview training in adults with autism spectrum disorder. J Autism Dev Disord, 44, 2450–2463. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Smith MJ, Ginger EJ, Wright M, Wright K, Boteler Humm L, Olsen D, Bell MD, & Fleming MF (2014b). Virtual reality job interview training for individuals with psychiatric disabilities. J Nerv Ment Dis, 202, 659–667. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Vinogradov S, Fisher M, & de Villers-Sidani E (2012). Cognitive training for impaired neural systems in neuropsychiatric illness. Neuropsychopharmacology, 37, 43–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. White WL (2009). The mobilization of community resources to support long-term addiction recovery. J Subst Abuse Treat, 36, 146–158. [DOI] [PubMed] [Google Scholar]
  40. Wilkinson GS, & Robertson GJ (2006). Wide Range Achievement Test 4 Professional Manual. Lutz, FL: Psychological Assessment Resources. [Google Scholar]
  41. Zemore SE, Kaskutas LA, & Ammon LN (2004). In 12-step groups, helping helps the helper. Addiction, 99, 1015–1023. [DOI] [PubMed] [Google Scholar]

RESOURCES