Abstract
Objective
Individuals with severe mental illness have low employment rates and the job interview presents a critical barrier for them to obtain competitive employment. Prior randomized controlled trials (RCTs) indicated virtual reality job interview training (VR-JIT) improved job interviewing skills among trainees. The current study assessed whether VR-JIT participation was associated with greater odds of receiving job offers upon 6-month follow-up.
Methods
VR-JIT trainees (n=39) and a comparison group (n=12) completed a brief survey approximately 6 months after finishing the RCTs to assess the efficacy of VR-JIT. Primary vocational outcome measures included receiving a job offer and number of weeks searching for employment.
Results
A higher proportion of trainees received a job offer than comparison participants (51% (n=20) vs. 25% (n=3), respectively). Trainees were more likely to receive a job offer than comparison participants (odds ratio=9.64, p<.05) after accounting for cognition, recency of last job, and diagnosis. Among trainees, there were greater odds of receiving a job offer for each completed VR-JIT trial (odds ratio=1.41, p<.05) and a greater number of completed VR-JIT trials predicted fewer weeks searching for employment (β=−.74, p<.05).
Conclusions
Results provide preliminary support that VR-JIT is a promising intervention associated with enhanced vocational outcomes among individuals with severe mental illness. Given that participants had minimal access to standardized vocational services, future research could evaluate the effectiveness of VR-JIT among individuals with and without access to standardized vocational services as well as evaluate strategies to implement VR-JIT within a large community mental health service provider.
Keywords: virtual reality training, job interview skills, vocational training, vocational outcomes, severe mental illness, PTSD
Introduction
Individuals with severe mental illnesses including bipolar and major depressive disorders, and schizophrenia as well as military veterans with post-traumatic stress disorder (PTSD) experience disabling symptoms that hinder their vocational outcomes (1, 2). Although approximately 60% of individuals receiving mental health services want to work and may actively seek employment (3, 4), they have low employment rates. Specifically, only 22-38% of individuals with severe mental illness (including veterans with PTSD) are competitively employed (5-7).
One factor that may contribute to a low employment rate is that individuals with severe mental illness have social deficits that may interfere with successfully navigating the job interview (8-12). Thus, these individuals may have difficulty translating their life experiences and/or military training into work-related skills within a community setting. Hence, we developed a virtual reality training program to help facilitate learning job interviewing skills (13-15). Virtual Reality Job Interview Training (VR-JIT) provides trainees with the opportunity to engage in repeated interviewing trials with a virtual human resources representative, and learn from didactic online materials.
Two randomized controlled trials (RCTs) assessed the efficacy of VR-JIT, compared to a waitlist group, within cohorts of individuals with severe mental illness (13) and veterans with PTSD (14). Approximately 37% of participants were currently or previously enrolled in vocational services, but not in supported employment. The studies were administered by the same research team and used identical procedures. The participants were informed they could complete up to 10 hours of training across 5 sessions. Each simulated interview trial lasted 20-30 minutes and trainees completed approximately 15 trials. The training followed a hierarchical learning framework where trainees advanced from easy to medium to hard levels based on their performance, and the results suggested VR-JIT enhanced interviewing skills (13, 14). However, the utility of the training would be further validated if using VR-JIT was associated with receiving job offers after completing real-life interviews.
In this study, 6-month follow-up data were collected from participants who completed the efficacy studies. First, we hypothesized that the increased interviewing self-confidence observed in trainees during the efficacy study would be maintained at 6-month follow-up. Second, we hypothesized that trainees would have greater odds of receiving a job offer compared to waitlist comparison participants. Third, we hypothesized that a larger number of completed VR-JIT trials, a greater VR-JIT performance slope, and increased self-confidence would predict greater odds of receiving a job offer. Fourth, we hypothesized that a larger number of completed VR-JIT trials, a greater VR-JIT performance slope, and increased self-confidence would predict fewer weeks searching for employment. Lastly, we hypothesized that after six months, trainees would report VR-JIT helped them find employment. We evaluated neurocognition and recent employment history as covariates in the analyses as they are known predictors of vocational outcomes in this population (16-18).
Methods
Participants
Participants included 70 individuals with severe mental illness or United States military veterans who completed prior efficacy studies of VR-JIT (13, 14). For inclusion into the efficacy studies, non-veteran participants were required to have a diagnosis of bipolar disorder, major depressive disorder, schizophrenia or schizoaffective disorder; while veterans were eligible for the study if they were diagnosed with PTSD and a co-morbid mood or psychotic disorder. Details on participant recruitment are reported elsewhere (13, 14). Bachelor’s or Ph.D.-level research staff confirmed diagnoses using the Mini-International Neuropsychiatric Interview (19).
Inclusion criteria included: 18-65 years old, minimum 6th grade reading level using the Wide Range Achievement Test-IV (20), willingness to be video-recorded, unemployed or underemployed, and actively seeking employment. Exclusion criteria included having: a medical illness that significantly comprised cognition (e.g., Traumatic Brain Injury), an uncorrected vision or hearing problem, or current substance abuse or dependence. The Blinded University Institutional Review Board approved the study protocol, and all participants provided informed consent.
Participants from both efficacy studies were re-contacted after 6 months and asked to complete a follow-up survey. Seventy participants completed the original efficacy studies. Supplemental Figure 1 displays the flow of participants from the efficacy studies to the follow-up study. All comparison participants were informed they could return to use VR-JIT after the study and were reminded of this option during their last visit. Six waitlist comparison participants completed VR-JIT after the efficacy study and transitioned to the VR-JIT group for 6-month follow-up. However, one was lost to attrition. The final sample included n=39 VR-JIT participants and n=12 comparison participants who chose not to use VR-JIT.
Intervention
Virtual Reality Job Interview Training (VR-JIT) is a software application developed by Blinded Company. VR-JIT includes educational content about finding employment, an interactive role-play simulator, and integrated feedback. VR-JIT provides the opportunity to repeatedly practice interviews in a risk-free environment for individuals with a range of disabilities. During each virtual interview, Molly Porter, a human resources manager at a large department store asks questions about skills and experiences. Molly selects questions from 1,200 options to tailor each virtual interview based on customizable information (e.g., military history, gaps in employment), skill level, and responses. This variation allows trainees to practice until they master the skills at three difficulty levels and gain confidence to successfully interview for employment. Images of Molly and the VR-JIT interface can be found at [Blinded Website]. Additional methodological details are reported elsewhere (13, 14).
Study Procedures
Research staff contacted participants weekly beginning 6 months after completing the efficacy studies to complete the follow-up survey using phone and email. If participants were unreachable then a recruitment letter was sent to their last known address. The efficacy studies and 6-month follow-up occurred between January 2013 and January 2014.
Baseline Study Measures
Participant Characteristics
The participants’ demographics (e.g., age), and vocational history (e.g., months since prior employment) were obtained via self-report.
Cognitive Measures
The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) (21) was administered to assess neurocognition. The total index score of the RBANS reflects: immediate memory, visuospatial capacity, language, attention, and delayed memory.
We assessed emotion recognition as a measure of basic social cognition using the Bell-Lysaker Emotion Recognition Task (22). We used an emotional perspective-taking task as a measure of advanced social cognition (23). Accuracy ratings for each task were generated using the number of correct responses.
Process Measures
Participants’ VR-JIT performance scores and the number of trials completed with VR-JIT were recorded. The software scored each simulated interview from 0-100 using an algorithm based on the appropriateness of trainee responses throughout the interview in the following domains: conveying oneself as a hard worker, sounding easy to work with, sharing things in a positive way, sounding honest, sounding interested in the position, acting professionally, negotiation skills, and establishing overall rapport with the interviewer. We evaluated VR-JIT performance score improvement across trials as a process measure by computing a linear regression ‘performance score’ slope for each participant based on the regression of their performance scores on the log of trial number.
Six-Month Follow-up Measures
The survey included a series of questions that asked participants to reflect on the prior 6 months and to report the total number of weeks they searched for employment, whether or not they completed a job interview (0=no, 1=yes) (and if so, how many), and whether or not they received (and accepted) a job offer (0=no, 1=yes).
The survey included the same self-confidence items that were collected during the efficacy studies (13, 14). Participants rated their self-confidence at performing interviews using a 7-point Likert scale to answer nine questions, with higher scores reflecting more positive views (e.g., “How comfortable are you going on a job interview?”). Total scores at the end of the two-week follow-up for the efficacy study and 6-month follow-up were computed. The internal consistencies at two-week follow-up (α=.91), and 6-month follow-up (α=.86) across all participants were strong.
The 6-month survey also assessed whether trainees felt VR-JIT helped them improve their interview skills, prepared them for interviews, helped them attain employment, gave them more confidence to interview, and using the training again would help them prepare for future interviews. Item ratings ranged from 1=extremely uncomfortable/skilled to 7=extremely comfortable/skilled.
Data Analysis
Chi-square and analyses of variance (ANOVA) were conducted using participant baseline characteristics to examine between-group differences. To evaluate whether improved self-confidence observed during the efficacy trial was maintained at 6-months, we conducted a paired sample t-test between the two-week and 6-month follow-up scores.
We conducted a stepwise logistic regression with job offer as the dependent variable to evaluate whether VR-JIT trainees had greater odds of receiving a job offer. Step 1 included neurocognition, the number of months since prior employment, and PTSD diagnosis, while VRJIT group status was added in Step 2.
Among VR-JIT trainees, we conducted a stepwise logistic regression with job offer as the dependent variable to evaluate whether VR-JIT process measures and self-confidence contributed to higher odds of receiving a job offer. Step 1 included neurocognition, the number of months since prior employment, and PTSD diagnosis. Step 2 included the number of completed VR-JIT trials, VR-JIT performance slope, and self-confidence at 6-month follow-up. For both logistic regressions, odds ratios (OR) and 95% confidence intervals (CI) were generated for each step. Nagelkerke R2 was computed to determine the proportion of explained variance.
Among VR-JIT trainees, we conducted a hierarchical linear regression, with the number of weeks searching for employment as the dependent variable, to evaluate whether VR-JIT process measures and self-confidence were associated with fewer weeks searching for employment. Step 1 included neurocognition, the number of months since prior employment, and PTSD diagnosis. Step 2 included the number of completed VR-JIT trials, VR-JIT performance slope, and self-confidence at 6-month follow-up.
Lastly, we used descriptive statistics to evaluate whether participants found VR-JIT to be helpful to them.
Results
Participant Characteristics
Baseline characteristics of the VR-JIT and comparison groups did not differ with respect to their demographics and clinical, cognitive, and vocational histories (all p>.10) (Table 1). A greater proportion of participants with PTSD were observed in the comparison group compared to the VR-JIT group at a trend level (p=.09). As a result, we evaluated PTSD as a fixed-effect covariate in the ANOVAs and regressions.
Table 1.
Characteristics of study sample at 6-month follow-up
| Control Group (n=12) | VR-JIT Group (n=39) | χ 2 | T | df | p | |||
|---|---|---|---|---|---|---|---|---|
|
|
||||||||
| N | % | N | % | |||||
| Demographics | ||||||||
| Age (M±SD) | 49.1±10.9 | 47.0±12.4 | 0.5 | 49 | n.s | |||
| Gender (% male) | 6 | 50% | 29 | 74% | 2.5 | 1 | n.s | |
| Parental education (M±SD) | 12.8±3.5 | 13.5±2.7 | −0.7 | 49 | n.s | |||
| Race | n.s | |||||||
| % Caucasian | 5 | 42% | 13 | 33% | n.s | |||
| % African-American | 6 | 50% | 25 | 64% | 1.3 | 1 | n.s | |
| % other | 1 | 8% | 1 | 3% | n.s | |||
| Vocational history | n.s | |||||||
| Months since prior employment (M±SD) | 31.8±41.9 | 44.8±53.5 | −0.8 | 49 | n.s | |||
| Prior full-time employment (%) | 10 | 83% | 35 | 90% | 0.3 | 1 | n.s | |
| Prior paid employment (any type) (%) | 12 | 100% | 38 | 97% | 0.3 | 1 | n.s | |
| Prior/current enrollment in vocational training | 5 | 42% | 13 | 33% | 0.3 | 1 | n.s | |
| Cognitive function | n.s | |||||||
| Neurocognition (M±SD)a | 91.2±10.0 | 91.9±16.9 | −0.1 | 49 | n.s | |||
| Basic social cognition (M±SD)b | .69±.14 | .70±.14 | −0.2 | 49 | n.s | |||
| Advanced social cognition (M±SD)c | .78±.10 | .78±.09 | −0.2 | 49 | n.s | |||
| Clinical history | n.s | |||||||
| Posttraumatic stress disorder | 8 | 67% | 15 | 39% | 2.9 | 1 | .09 | |
| Major depressive disorder | 6 | 50% | 18 | 46% | 0.1 | 1 | n.s | |
| Bipolar disorder | 4 | 25% | 13 | 33% | 0.5 | 1 | n.s | |
| Schizophrenia or schizoaffective disorder | 1 | 8% | 7 | 18% | 0.6 | 1 | n.s | |
Neurocognition index scores ranged from 0 to 160 with higher scores indicating higher neurocognition;
Basic social cognition scores ranged from 0 to 1.0 with higher scores indicating higher basic social cognition;
Advanced social cognition scores ranged from 0 to 1.0 with higher scores indicating higher advanced social cognition.
Vocational Outcomes Among 6-Month Follow-up Participants
We found that a similar proportion of VR-JIT and comparison participants completed job interviews (before and after adjusting for a PTSD diagnosis) (p>.10) (Table 2). Although 51% of VR-JIT trainees attained a job offer compared to 25% of comparison participants, this difference only attained significance after adjusting for PTSD diagnosis (p=.04) (Table 2). The groups did not differ with respect to the adjusted and unadjusted proportion of participants who accepted job offers, the number of weeks they searched for employment, or the mean number of job interviews completed (all p>.10) (Table 2). A PTSD diagnosis was a significant covariate for whether or not an interview was completed (p<.05), but not for any other between-group differences.
Table 2.
Between-groupmean differences for 6-month follow-up outcomes
| Control Group (n=12) |
VR-JIT Group (n=39) |
|||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
|
||||||||||
| N | % | 95% CI | N | % | 95% CI | χ 2 | F | df | p | |
| Unadjusted Values | ||||||||||
| Total weeks looking for a job (M±SD) | 10.25±9.19 | 4.61-15.89 | 13.21±9.86 | 10.08-16.33 | .85 | 1,49 | ns | |||
| Total job interviews completed (M±SD) | 4.42±4.36 | 2.63-6.20 | 2.62±2.59 | 1.63-3.61 | 3.14 | 1,49 | .08 | |||
| Proportion of completed job interviews | 10 | 83 | 30 | 77 | .22 | 1 | ns | |||
| Proportion who received job offer | 3 | 25 | 20 | 51 | 2.56 | 1 | ns | |||
| Proportion who accepted job offer | 3 | 100 | 15 | 75 | .73 | 1 | ns | |||
| Adjusted Values | ||||||||||
| Total weeks looking for a job (M±SD) | 10.25±9.97 | 4.46-16.04 | 12.38±9.66 | 9.27-15.49 | .43 | 3,47 | ns | |||
| Total job interviews completed (M±SD) | 4.19±3.32 | 2.26-6.11 | 2.61±3.21 | 1.58-3.65 | 2.10 | 3,47 | ns | |||
| Proportion of completed job interviews (M±SD) | .75±.43 | .50-1.00 | .79±.42 | .65-.92 | .07 | 3,47 | ns | |||
| Proportion who received job offer (M±SD) | .19±.51 | .11-.49 | .54±.50 | .38-.70 | 4.42 | 3,47 | .04 | |||
| Proportion who accepted job offer (M±SD) | 1.00±.81 | .51-1.49 | .75±.57 | .56-.94 | 1.40 | 3,47 | ns | |||
Self-Confidence at 6 Months Among VR-JIT Participants
Mean self-confidence in interviewing skills did not differ between the post-intervention (51.9±7.9) and the 6-month follow-up (52.1±7.6) assessment within VR-JIT trainees (T=.17, df=33, p>.10). This analysis excluded the n=5 waitlist participants who completed VR-JIT.
Odds of Receiving a Job Offer
Step 1 of the logistic regression included neurocognition, months since prior employment, and PTSD diagnosis, and explained 40% of the variance in receiving a job offer. Step 2 added VR-JIT participation into the model, which explained a total of 52% of the variance in receiving a job offer (Table 3). Overall, Step 2 had 77% accuracy at predicting cases receiving a job offer, and the omnibus test of model coefficients was significant (X2=24.90, df=4, p<.001). Moreover, the odds of receiving a job offer were higher for VR-JIT trainees compared to comparison participants (OR=9.64, 95% CI=1.48-62.92), while participants had 4% reduced odds of receiving a job offer for each month since prior employment (OR=.96, 95% CI=.94-.99). A PTSD diagnosis was associated with greater odds of receiving a job offer at the trend level (p=.06), while neurocognition was non-significant (p>.10).
Table 3.
VR-JIT group status as a predictor of a job offer
| Step 1 | Step 2 | |||||
|---|---|---|---|---|---|---|
|
|
||||||
| OR | 95% CI | p | OR | 95% CI | p | |
| Step 1 | ||||||
| Neurocognition | 1.04 | .99-1.09 | ns | 1.04 | .98-1.11 | ns |
| Months since prior employment | .97 | .95-.99 | .01 | .96 | .92-.99 | .01 |
| PTSD diagnosis | 2.69 | .70-10.40 | ns | 4.68 | .97-22.59 | .06 |
| Step 2 | ||||||
| VR-JIT group (yes or no) | -- | -- | 9.64 | 1.48-62.92 | .02 | |
p<.10,
p<.05,
p<.01;
Step 1 Nagelkerke R2=.40;
Step 2 Nagelkerke R2=.52.
VR-JIT Trainees and Job Offers
Step 1 of the logistic regression included neurocognition, months since prior employment, and PTSD diagnosis, and explained 59% of the variance in receiving a job offer. Step 2 added total completed trials, performance slope, and self-confidence at follow-up, which explained a total of 68% of the variance in receiving a job offer (Table 4). Overall, Step 2 had 79.5% accuracy at predicting cases receiving a job offer, and the omnibus test of model coefficients was significant (X2=27.79, df=6, p<.001). For each completed VR-JIT trial, the odds were 1.41 times higher that participants received a job offer (OR=1.41, 95% CI=1.02-1.95, p<.05) (Table 4). Hence, the completion of each VR trial (15.1±5.5 total trials completed) was associated with 41% higher odds of receiving a job offer. VR-JIT trainees had 7% lower odds of receiving a job offer for each month since prior employment (OR=.93, 95% CI=.87-.98, p<.05). Neurocognition, PTSD diagnosis, performance slope, and self-confidence were non-significant (all p>.10) (Table 4).
Table 4.
VR-JIT processes as predictors of a job offer
| Step 1 | Step 2 | |||||
|---|---|---|---|---|---|---|
|
|
||||||
| OR | 95% CI | p | OR | 95% CI | p | |
| Step 1 | ||||||
| Neurocognition | 1.06 | .98-1.14 | ns | 1.06 | .98-1.16 | ns |
| Months since prior employment | .94* | .89-.99 | .02 | .93 | .87-.98 | .01 |
| PTSD diagnosis | 3.90 | .59-25.74 | ns | 3.33 | .44-25.56 | ns |
| Step 2 | ||||||
| Total VR trials completed | -- | -- | -- | 1.41 | 1.02-1.95 | .04 |
| VR-JIT performance slope | -- | -- | -- | 1.19 | .79-1.78 | ns |
| Self-confidence at follow-up | -- | -- | -- | .96 | .82-1.12 | ns |
Self-confidence scores ranged from 9 to 63 with higher scores indicating higher self-confidence.
p<.05;
Step 1 Nagelkerke R2=.59;
Step 2 Nagelkerke R2=.68.
VR-JIT Training and Searching for Employment
Step 1 of the linear regression included neurocognition, months since prior employment, and a PTSD diagnosis, but did not explain significant variance in the number of weeks searching for employment (F=1.02, df=3,16, p=.41) (Table 5). In Step 2, the addition of total completed trials, performance slope, and self-confidence at follow-up explained 37% of the variance in the number of weeks searching for employment (F=3.37, df=3,13, p=.05). A greater number of completed VR-JIT trials (B= −3.79, SE=1.37, p=.02) and a larger slope of VR performance scores (B= −1.37, SE=.73, p=.08), at the trend level, were associated with fewer weeks searching for employment. Neurocognition, months since prior employment, PTSD diagnosis, and self-confidence were non-significant predictors (all p>.10) (Table 5).
Table 5.
Regression analyses to determine VR-JIT predictors of weeks searching for employment (Standardized Beta)
| Step 1 | p | Step 2 | p | |
|---|---|---|---|---|
| Step 1 | ||||
| Neurocognition | .24 | ns | .03 | ns |
| Months since prior employment | −.04 | ns | .20 | ns |
| PTSD diagnosis | −.26 | ns | −.05 | ns |
| Step 2 | ||||
| Number of completed VR trials | -- | -- | −.74 | .02 |
| VR performance slope | -- | -- | −.45 | .08 |
| Self-confidence at follow-up | -- | -- | −.04 | ns |
Self-confidence scores ranged from 9 to 63 with higher scores indicating higher self-confidence;
Step 1 R2=.16;
Step 2 R2=.53.
VR-JIT Helpfulness
Approximately 90% of VR-JIT trainees strongly agreed or agreed that the training increased their confidence to go on interviews, the training helped them improve their interview skills, better prepared them for interviews, and they would like to use VR-JIT again to enhance their interviewing skills. Sixty-nine percent of trainees agreed or strongly agreed that VR-JIT helped them get a job (data not shown).
Discussion
Prior studies assessing the efficacy of VR-JIT during lab-based RCTs observed that trainees improved their interviewing skills after simulating job interviews with a virtual character (13, 14). The present study found that trainees had greater odds of receiving a job offer after interviewing for a position than comparison participants. Moreover, this study observed a dosing effect as the odds of getting a job offer increased by 1.4 times for every completed virtual interview. A dosing effect was also associated with a shorter duration of searching for a job (prior to obtaining an offer). Lastly, trainees demonstrated sustained improvement in their self-confidence between the RCT and 6-month follow-up. Additionally, most trainees reported that VR-JIT helped them prepare for interviews and attain a job offer.
There are several directions to continue the evaluation of VR-JIT. Only 37% of participants were receiving vocational services at entry into the efficacy studies, and were not previously enrolled in supported employment. Hence, it would be important for future research to evaluate the impact of VR-JIT on larger study samples with and without access to vocational services. This is an important question as not all individuals with severe mental illness have access to evidence-based vocational services. Also, a recent report suggests that research is needed to study technological advances to standardized vocational services (24). Thus, an equally important area for future research would be to evaluate whether VR-JIT can enhance vocational outcomes for individuals using evidence-based vocational services (e.g., Individual Placement and Support) (25, 26) and to evaluate strategies to implement VR-JIT within a large community mental health service provider.
There were limitations to the study that must be considered. The sample size was relatively small and only retained 71% (n=34 of 48) of the VR-JIT participants and 55% (n=12 of 22) of the comparison, and 22% (n=5 of 22) of the comparison participants who transitioned to the VR-JIT group. We did not record data on the types of jobs attained and the pay received. Moreover, we did not directly assess participants’ motivation to find a job. Thus, the transfer of 5 participants from the waitlist to the training group could reflect lower motivation in participants who did not use VR-JIT. However, we observed that comparison participants did not differ from VR-JIT trainees with respect to the proportion that completed a job interview, the number of interviews completed, and the number of weeks searching for a job during the 6-month follow-up period. Lastly, we did not assess the types of vocational services received during the 6-month follow-up.
The study had notable strengths to help overcome the limitations. First, during the efficacy studies, the participants were randomized to VR-JIT or a wait-list group. This approach enabled our follow-up of a larger sample of trainees who used VR-JIT after the efficacy study. In an effort to address any bias associated with this approach, we conducted a post hoc analysis while excluding these 5 waitlist participants. The results showed VR-JIT was still associated with greater odds of receiving a job offer (Supplemental Table 1). The analysis of baseline characteristics revealed that the two groups completing the 6-month follow-up did not differ with respect to prior vocational training and cognition, and the primary analyses were conducted while controlling for known predictors of vocational outcomes (16-18). We evaluated VR-JIT among individuals who were motivated to actively seek employment, which is the target group most likely to use the intervention. The training appears to be effective across diagnoses, but larger and more focused community samples are needed to examine whether the training has diagnostic specificity.
Conclusions
In conclusion, VR-JIT trainees were more likely than comparison participants to receive job offers by 6-month follow-up. Moreover, greater odds of receiving a job offer were associated with the amount of training, while the amount of training and performance scores were associated with how quickly trainees received a job offer. Thus, VR-JIT is a promising intervention, and future studies can evaluate whether this training enhances vocational outcomes for individuals with severe mental illness with and without access to evidence-based services.
Supplementary Material
Acknowledgment
Dr. X and Ms. A are employed by and own shares in Blinded Company. They contributed to the manuscript, but were not involved in analyzing the data. Dr. Z was a paid consultant by Blinded Company to assist with the development of virtual reality job interview training. Dr. Z and his family do not have a financial stake in the Blinded Company. We would like to thank Dr. B for his consultation on the statistical analyses. The authors acknowledge research staff at Blinded University’s Clinical Research Program for data collection and our participants for volunteering their time. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.
Grant Support
Dr. X received a grant from the National Institute of Mental Health to develop virtual reality job interview training (R44 MH080496), and funds were subcontracted to Dr. Y at Blinded University to complete the study.
Footnotes
Disclosures
The remaining authors report no conflicts of interest outside of their salary support to complete the study.
Contributor Information
Matthew J Smith, Northwestern University Feinberg School of Medicine - Department of Psychiatry and Behavioral Sciences 446 E. Ontario Suite 1000 , Chicago, Illinois 60611 matthewsmith@northwestern.edu.
Michael F Fleming, Northwestern University Feinberg School of Medicine - Psychiatry and Behavioral Sciences Chicago, Illinois.
Michael A. Wright, Northwestern University Feinberg School of Medicine - Psychiatry and Behavioral Sciences Chicago, Illinois
Neil Jordan, Northwestern University - Psychiatry & Behavioral Sciences 710 N Lake Shore Dr Ste 904 , Chicago, Illinois 60611.
Laura Boteler Humm, SIMmersion, LLC. Columbia, Maryland.
Dale Olsen, SIMmersion, LLC. Columbia, Maryland.
Morris D. Bell, Yale University School of Medicine - Psychiatry Psychology Service 116B 950 Campbell Avenue , West Haven, Connecticut 06516
References
- 1.Van Til L, Fikretoglu D, Pranger T, et al. Work reintegration for veterans with mental disorders: a systematic literature review to inform research. Phys Ther. 2013;93(9):1163–1174. doi: 10.2522/ptj.20120156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Bell MD, Lysaker PH. Psychiatric symptoms and work performance among persons with severe mental illness. Psychiatr Serv. 1995;46(5):508–510. doi: 10.1176/ps.46.5.508. [DOI] [PubMed] [Google Scholar]
- 3.Frounfelker RL, Wilkniss SM, Bond GR, et al. Enrollment in supported employment services for clients with a co-occurring disorder. Psychiatr Serv. 2011;62(5):545–547. doi: 10.1176/ps.62.5.pss6205_0545. [DOI] [PubMed] [Google Scholar]
- 4.Ramsay CE, Broussard B, Goulding SM, et al. Life and treatment goals of individuals hospitalized for first-episode nonaffective psychosis. Psychiatry Res. 2011;189(3):344–348. doi: 10.1016/j.psychres.2011.05.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Luciano A, Meara E. Employment status of people with mental illness: national survey data from 2009 and 2010. Psychiatr Serv. 2014;65(10):1201–1209. doi: 10.1176/appi.ps.201300335. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Mechanic D, Blider S, McAlpine DD. Employing persons with serious mental illness. Health Aff (Millwood) 2002;21(5):242–253. doi: 10.1377/hlthaff.21.5.242. [DOI] [PubMed] [Google Scholar]
- 7.Zivin K, Bohnert AS, Mezuk B, et al. Employment status of patients in the VA health system: implications for mental health services. Psychiatr Serv. 2011;62(1):35–38. doi: 10.1176/ps.62.1.pss6201_0035. [DOI] [PubMed] [Google Scholar]
- 8.Couture SM, Penn DL, Roberts DL. The functional significance of social cognition in schizophrenia: a review. Schizophr Bull. 2006;32(Suppl 1):S44–63. doi: 10.1093/schbul/sbl029. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Dickinson D, Bellack AS, Gold JM. Social/communication skills, cognition, and vocational functioning in schizophrenia. Schizophr Bull. 2007;33(5):1213–1220. doi: 10.1093/schbul/sbl067. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lahera G, Ruiz-Murugarren S, Iglesias P, et al. Social cognition and global functioning in bipolar disorder. J Nerv Ment Dis. 2012;200(2):135–141. doi: 10.1097/NMD.0b013e3182438eae. [DOI] [PubMed] [Google Scholar]
- 11.Samame C. Social cognition throughout the three phases of bipolar disorder: A state-of-the-art overview. Psychiatry Res. 2013;210(3):1275–1286. doi: 10.1016/j.psychres.2013.08.012. [DOI] [PubMed] [Google Scholar]
- 12.Mazza M, Giusti L, Albanese A, et al. Social cognition disorders in military police officers affected by posttraumatic stress disorder after the attack of An-Nasiriyah in Iraq 2006. Psychiatry Res. 2012;198(2):248–252. doi: 10.1016/j.psychres.2011.11.027. [DOI] [PubMed] [Google Scholar]
- 13.Blinded Author Publication
- 14.Blinded Author Publication
- 15.Blinded Author Publication
- 16.Burke-Miller JK, Cook JA, Grey DD, et al. Demographic characteristics and employment among people with severe mental illness in a multisite study. Community Ment Health J. 2006;42(2):143–159. doi: 10.1007/s10597-005-9017-4. [DOI] [PubMed] [Google Scholar]
- 17.Catty J, Lissouba P, White S, et al. Predictors of employment for people with severe mental illness: results of an international six-centre randomised controlled trial. Br J Psychiatry. 2008;192(3):224–231. doi: 10.1192/bjp.bp.107.041475. [DOI] [PubMed] [Google Scholar]
- 18.Gold JM, Goldberg RW, McNary SW, et al. Cognitive correlates of job tenure among patients with severe mental illness. Am J Psychiatry. 2002;159(8):1395–1402. doi: 10.1176/appi.ajp.159.8.1395. [DOI] [PubMed] [Google Scholar]
- 19.Sheehan DV, Lecrubier Y, Sheehan KH, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59(Suppl 20):22–33. quiz 34-57. [PubMed] [Google Scholar]
- 20.Wilkinson GS, Robertson GJ. Wide Range Achievement Test 4 Professional Manual. Psychological Assessment Resources; Lutz, FL: 2006. [Google Scholar]
- 21.Randolph C, Tierney MC, Mohr E, et al. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): preliminary clinical validity. J Clin Exp Neuropsychol. 1998;20(3):310–319. doi: 10.1076/jcen.20.3.310.823. [DOI] [PubMed] [Google Scholar]
- 22.Bell M, Bryson G, Lysaker P. Positive and negative affect recognition in schizophrenia: a comparison with substance abuse and normal control subjects. Psychiatry Res. 1997;73(1-2):73–82. doi: 10.1016/s0165-1781(97)00111-x. [DOI] [PubMed] [Google Scholar]
- 23.Smith MJ, Horan WP, Cobia DJ, et al. Performance-based empathy mediates the influence of working memory on social competence in schizophrenia. Schizophrenia Bulletin. 2014;40(4):824–834. doi: 10.1093/schbul/sbt084. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Lord SE, McGurk SR, Nicholson J, et al. The potential of technology for enhancing individual placement and support supported employment. Psychiatr Rehabil J. 2014;37(2):99–106. doi: 10.1037/prj0000070. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Bond GR, Drake RE, Becker DR. An update on randomized controlled trials of evidence-based supported employment. Psychiatr Rehabil J. 2008;31(4):280–290. doi: 10.2975/31.4.2008.280.290. [DOI] [PubMed] [Google Scholar]
- 26.Drake RE, Bond GR. IPS Supported Employment: A 20 year Update. American Journal of Psychiatric Rehabilitation. 2011;14(3):155–164. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.