Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Feb 1.
Published in final edited form as: Early Interv Psychiatry. 2019 Jun 10;14(1):106–114. doi: 10.1111/eip.12831

Virtual Reality Assessment of Functional Capacity in the Early Course of Schizophrenia: Associations with Cognitive Performance and Daily Functioning

Joseph Ventura 1, Tamara Welikson 2, Arielle Ered 3, Kenneth L Subotnik 1, Richard SE Keefe 4, Gerhard S Hellemann 1, Keith H Nuechterlein 1,5
PMCID: PMC6901817  NIHMSID: NIHMS1035812  PMID: 31183960

Abstract

Introduction

Computer-based virtual reality assessments of functional capacity have shown promise as a reliable and valid way to assess individuals with multi-episode schizophrenia. However, there has been little research utilizing this innovative approach with young patients who are in the early phase of schizophrenia.

Methods

Outpatients in the early course of schizophrenia (n=42) were compared to controls (n=13) at cross-sectional study points. Patients were within 2 years of their first psychotic episode, were an average of 22.2 years old and had an average of 12.3 years of education. We used the Virtual Reality Functional Capacity Assessment Tool (VRFCAT) and the UCSD Performance-Based Skills Assessment-2 (UPSA-2) to assess functional capacity. The MATRICS Consensus Cognitive Battery (MCCB) and the Cognitive Assessment Interview (CAI) were the measures of cognitive functioning. The Global Functioning Scale: Role (GFS-R) and Social (GFS-S), and Role Functioning Scale (RFS) were the measures of daily functioning.

Results

Early course patients vs. controls were slower (Patient M=830.41secs vs. Control M=716.84 secs; t=3.0, p<.01) and committed more errors (Patient M=3.2 vs Control M=1.7, t=2.9, p<.01) on the VRFCAT. Total Time was significantly correlated with the UPSA (r=−.66, p<.01), MCCB (r=−.70, p<.01), CAI (r=−.51, p<01), and GFS role (r=−.52, p<.01) and social functioning (r=−.43, p=.03).

Discussion

We extend previous findings to patients with first-episode schizophrenia. Virtual reality based performance was correlated with a standard test of functional capacity, indicating VRFCAT validity. Furthermore, correlations with cognitive functioning and occupational/school and social functioning indicate promise as a co-primary measure to track changes in response to treatment.

Keywords: Schizophrenia, Virtual Reality, Functional Capacity, First episode, MATRICS, Assessment Neurocognition, Cognitive Assessment, Daily functioning

Introduction

The MATRICS project consensus group made very specific recommendations to the US Food and Drug Administration (FDA) about which standard cognitive measures would reliably and validly measure cognitive change and developed a battery of tests for this purpose known as the MATRICS Consensus Cognitive Battery (Nuechterlein et al., 2008). However, when considering the approval criteria for new medications, the US FDA required that potential cognitive enhancing drugs demonstrate not only cognitive performance improvements but also evidence of improvements in the patient’s functioning (Laughren, 2001). However, a change in real-world functioning was not be a requirement of drug approval because changes in real-world functioning might not be observed in the relatively short-term periods that characterize most clinical trials. Therefore, the MATRICS group recommended that clinically meaningful change could be demonstrated through measures that simulate real-world functional capacity (Green et al., 2008). Functional capacity assessments are considered to be co-primary measures used to determine the efficacy of pharmacological and non-pharmacological treatments for improving cognition.

Functional capacity is defined as the abilities that are essential for an individual to function independently in a variety of community settings including work, school, and social situations such as with friends or family. In schizophrenia, functional capacity is a well-studied concept that has been measured objectively most often with the UCSD Performance Skills Assessment (UPSA) (Patterson et al., 2001). The UPSA has good psychometric properties, even in various versions such as the UPSA-Brief (Becattini-Oliveira et al., 2018). Clearly, functional capacity abilities have been shown to be very deficient in schizophrenia patients (Bowie et al., 2006). Interestingly, functional capacity has been found to be a mediator of the relationship between neurocognitive ability and real-world functioning (Bowie and Harvey, 2008; Bowie et al., 2006). However, in two studies that examined the UPSA and quality of life (Green et al., 2011; Narvaez et al., 2008) there were discrepancies in the strength of the correlations (r=0.15 and r=0.46) and inconsistent findings for predicting functioning in first episode patients (Vesterager et al., 2012). Furthermore, the UPSA is usually administered in clinical settings by personnel requiring training and a cumbersome set of test props, several of which are outdated in today’s technologically driven world, e.g., a push-button desktop telephone rather than a mobile phone. The outdated nature of the UPSA is even more apparent when assessing samples of young, early course patients. More needs to be done to develop innovative ways to assess functional capacity and to understand the relationship between functional capacity and daily functioning in individuals with early phase schizophrenia.

The assessment of neurocognitive functioning is largely standardized through the use of objective, performance-based neuropsychological instruments such as the MCCB. However, reliable and valid alternative methods of assessing functional capacity, in addition to the UPSA, are still being developed and investigated. Some approaches to the assessment of functionally-relevant cognitive impairment rely on subjective reports from patients and informants such as the SCoRS (Keefe et al., 2004) and the CAI (Ventura et al., 2010). Caregiver and informant reports might be needed and can provide valid information about an individual’s functioning. However, caregivers or informants are not always available and obtaining their input requires an extra step in the assessment process (Sabbag et al., 2011). Given increasing interest in clinical trials for demonstrating treatment effects in schizophrenia that extend beyond symptom relief, especially in early course patients, there is a growing need for easily administered and valid measures of functional capacity with improved sensitivity to treatment response and strong relationships to functional outcome.

Reliable evaluation of both cognitive performance and functional capacity are critical to the effective assessment and prediction of outcomes in individuals diagnosed with schizophrenia. Research using the Virtual Reality Assessment of Functional Capacity Tool (VRFCAT) has shown promise as a reliable and valid computer-based method of assessing cross-sectional assessment and change in cognition in patients with multi-episode schizophrenia (Keefe et al., 2016). There is a need to develop short, standardized, easy to administer assessment procedures across all phases of illness. However, whether the VRFCAT is also applicable in the early phase of schizophrenia is unknown. Yet, there is a strong interest in developing reliable and valid tools to assess change in functional capacity in patients with early schizophrenia who are receiving new pharmacological and non-pharmacological treatment approaches. To pursue this line of research, we need to determine whether the VRFCAT deficit found in individuals with multi-episode schizophrenia is also present in early course patients and whether VRFCAT performance is related to cognition, other functional capacity measures, and everyday functioning.

Studies examining the prediction of functional outcome in schizophrenia have suggested that everyday activities, such as social functioning, familial interactions, and school / work outcomes might be strongly influenced by cognitive skill and abilities (Green et al., 2000). Studies in the early course of illness have corroborated what has been found in advanced stages of illness (Nuechterlein et al., 2011). Specifically, deficits in cognition appear to predict work and school outcomes, usually better than social outcomes (Strassnig et al., 2015). In addition, interview-based measures of cognitive functioning have been shown to be associated with daily functioning (Ventura et al., 2010). The UPSA, the most popular cognitive capacity measure, is also significantly related to everyday functioning (Bowie et al., 2006). More needs to be done to determine if computer-based virtual reality methods, such as the VRFCAT, which is correlated with cognition as indexed by the MCCB, are equally as predictive as standard functional capacity approaches. Further, whether these recently developed and innovative approaches might be useful in the differential prediction of different domains of functioning, such as work functioning vs. social functioning, needs to be examined.

In this study of functional capacity in the early course schizophrenia patients we aimed to determine: 1) whether functional capacity deficits as measured by the VRFCAT are also present in first-episode schizophrenia patients, and 2) whether VRFCAT performance correlates with the UPSA, measures of cognitive functioning, and daily functioning. We hypothesized that the VRFCAT would be sensitive to deficits in functional capacity in individuals with first-episode schizophrenia. Also, we hypothesized that we would find associations between VRFCAT performance and standard functional capacity assessments as well as cognitive performance deficits and daily functioning.

Methods

Subjects

The sample consisted of 42 first-episode schizophrenia patients and 13 healthy controls who were demographically comparable (Table 1; also see (Nuechterlein et al., 2014)). All patients had an initial onset of psychosis within 2 years of the VRFCAT assessment date. In fact, most patients had a very recent psychotic episode onset averaging less than 9 months prior to the study assessments, M=8.6 (SD=5.8) months (range:1–24 months). All patients met DSM-IV criteria for schizophrenia, schizoaffective disorder depressed type, or schizophreniform disorder based on the Structured Clinical Interview for DSM-IV (SCID-IV). Exclusion criteria were (1) evidence of a neurological disorder, (2) evidence of significant and habitual drug abuse or alcoholism in the 6 months prior to hospitalization or of substance use that triggered the psychotic episode, and (3) estimated IQ <70.

Table 1.

Demographic Distribution of the Sample and Statistical Analysis of the Differences between Patients and Control Subjects

Patients (n=42) Mean(SD) [range] Controls (n= 13)Mean(SD) [range] t-statistic, p value
Age 23.02 (4.04) [18–35] 21.92 (1.61) [19–24] t = 1.44, 0.16
Education 13.04 (1.63) [11–16] 14.46 (1.27) [12–16] t = −2.86, 0.01
Parental Education 14.38 (4.24) [2–21] 13.08 (2.96) [7–18] t = 1.00, 0.32
Sex 31 (74%) Male 4 (31%) Male X2(1) = 7.75, 0.01
Race X2(3) = 4.57, 0.21
 Caucasian 19 (47.5%) 4 (30.8%)
 Asian 1 (2.5%) 2 (15.4%)
 African American 11 (27.5%) 2 (15.4%)
 Mixed 9 (22.5%) 5 (38.5%)
Ethnicity X2(1) = 1.11, 0.29
 Hispanic 12 (30.0%) 6 (46.2%)
 Non-Hispanic 28 (70.0%) 7 (53.8%)
Diagnosis ------- ---------
 Schizophrenia 28 (66%)
 Schizophreniform 10 (24%)
 Schizoaffective 4 (10%)
Age of Onset 21.5 (4.2) [16–35] ------- ---------
BPRS1 Symptoms ------- ---------
 Reality Distortion 2.7 (1.6) [1.0–5.5]
 Disorganization 1.6 (0.6) [1.0–3.3]
 Negative Symptoms 2.6 (1.0) [1.0–5.0]
 Depression 1.7 (0.7) [1.0–2.3]
Open in a new tab
1

Brief Psychiatric Rating Scale

All patients were enrolled in the UCLA Aftercare Research Program, an outpatient clinic that provides treatment in the form of antipsychotic medication, individual case management, psychoeducation, family education, and group and individual therapy focused on recovery and practical life skills. All patients were assessed at a cross-sectional point in time at or near baseline after they were on a stable outpatient dose of oral risperidone for at least three months. Control subjects received extensive screening with the SCID-IV and symptom rating scales to verify the absence of a major Axis I (DSM-IV) psychiatric disorder or schizophrenia-spectrum personality disorder, family history of psychosis, lifetime substance dependence, and abuse in the past 6 months of alcohol or substances. All participants gave written informed consent prior to data collection.

Procedures

Functional Capacity Assessment

UCSD Performance-based Skills Assessment-Version 2 (UPSA-2) is a functional capacity measure of five general skills that were previously identified as essential to functioning in the community: organization/planning, finance, communication, transportation, household management, and an additional medication management ability assessment (Patterson et al., 2001; Patterson et al., 2002). The UPSA-2 involves role-play tasks that are simulations of situations that the person may encounter in the community. Higher scores indicated better performance. The dependent variable was the total score.

Virtual Reality Functional Capacity Assessment Tool (VRFCAT) was the computer-based measure of functional capacity (Keefe et al., 2016; Ruse et al., 2014). In an attempt to enhance the assessment of functional capacity beyond the UPSA’s paper-n-pencil version, researchers developed a computer screen-based experience that generates a three dimensional image that appears to surround the user. The VRFCAT uses mini-scenarios to measure the participant’s ability to complete four different functional capacity tasks: checking the availability of items to complete a recipe and determining which items are needed, taking a “virtual” bus trip to the grocery store to obtain the needed items which requires selecting the correct bus and using exact change to pay the fare, shopping in the grocery store and managing currency for the transportation and food purchases. All participants received a brief tutorial which included sample practice tasks similar to those from the VRFCAT. The dependent variables were: 1) Time to Completion, 2) Number of Errors, and 3) Number of Forced Progressions. For all tasks, participants who were unable to complete a specific task within the specified time period were given a Time to Completion score of 300 for that task and were automatically progressed to the next task.

Assessment of Cognitive Functioning

MATRICS Consensus Cognitive Battery (MCCB) was the objective measure of cognitive functioning (Nuechterlein et al., 2008). An Overall Composite score was derived from the seven MATRICS domains of cognitive functioning (Nuechterlein et al., 2004): Speed of Processing, Attention/vigilance, Working Memory, Verbal Memory, Reasoning and Problem Solving, and Social Cognition. The age and gender corrected T-score for the Cognitive Composite was used for these analyses.

Cognitive Assessment Interview (CAI) is an interview-based assessment of cognitive functioning which includes 10 items that assess 6 of the 7 MATRICS cognitive domains: Speed of Processing, Attention/vigilance, Working Memory, Verbal Memory, Reasoning and Problem Solving, and Social Cognition (Ventura et al., 2010). The CAI is a reliable and valid assessment that was administered to the patient and an informant who was required to know the patient well enough to comment on his or her cognitive functioning (Ventura et al., 2016). CAI items were rated on a seven-point scale with defined anchor points referenced to healthy people of similar educational and socio-cultural background. Higher scores reflect more severe cognitive deficits that impact everyday functioning. The CAI Global Rater score based on patient and informant information was used in the correlation analyses.

Functional Outcome Assessment

Global Functioning Scale-Role (GFS-R) and Global Functioning Scale-Social (GFS-S) was used to assess functional outcome. The Global Functioning Scale-Role (GFS-R) (Cornblatt et al., 2007; Niendam et al., 2006), a 10-point rating scale with well-defined behavioral anchors developed for evaluating school and job functioning in young adults and the Global Functioning Scale-Social (GFS-S) measures overall quality and quantity of social interactions. The GFS and the RFS were rated by a trained staff member. The dependent variable was the mean score for the Role Functioning and the mean score for the Social Functioning (range 1–10), with higher scores indicating better functioning.

Role Functioning Scale (RFS) was used to assess functional outcome for the following domains: Independent Living, Work Productivity, Family Relationships, and Social Relationships (Goodman et al., 1993; Green and Gracely, 1987). Ratings were based on a semi-structured interview using standardized probe questions. The items on the RFS are anchored 1–7, such that higher scores reflect decreasing reliance on agency-related support and increasing independence in community functioning. We used the global score for each of the four separate functional domains.

Statistical Analysis

Independent t-tests were conducted to examine if there were statistically significant differences in how early course schizophrenia patients performed on the VRFCAT relative to control subjects of similar age that were matched on parental education. Raw scores for the MCCB subtests were converted to T-scores and an MCCB overall composite score was created through the use of the MCCB scoring program. Scores on the UPSA were standardized with a range of 0 to 100. Correlational analyses using Pearson correlation coefficients were used to examine relationships among these measures for the patients only as we were interested in the relationships between the VRFCAT and cognition, and VRFCAT with daily functioning.

Results

Sample Demographics

The sample consisted of 48 patients who were in the early course of schizophrenia and 13 healthy controls. Statistical test comparisons confirmed that the two samples were similar in age, race, and ethnic background (Table 1). However, the patients were more likely to be female, as expected in a research sample of schizophrenia patients.

Comparisons of VRFCAT Functional Capacity between Patients and Controls

We confirmed that the deficit in functional capacity performance as measured using VRFCAT is present in the early course of schizophrenia. Patients as compared to control subjects were slower (Patient M=830.41 secs vs. Control M=716.84 secs; t=3.0, p<.01) and committed more errors (Patient M=3.2 vs Control M=1.7; t=2.9, p<.01). However, there was no statistically significant difference when comparing patients vs. controls for the VRCAT variable Number of Forced Progressions (M=.24 vs M=.08, t=1.2, p=.21). In addition, there was no indication that the sex of the participants played a significant role in their performance on the VRFCAT. There was no significant difference in the performance of women compared to men in Total Completion Time (Male M=814.89 secs vs. Women M=783.83 secs, t(53)=0.7, p=.45) and did not commit significantly Total Number of Errors (Men M=2.8 vs. Women M=2.6; t(53)=0.2, p=.86). Also, there was no significant difference when comparing men vs. women for Number of Forced Progressions (Men M=.20 vs. Women M=.20, t(53)=.00, p=1.00).

Another variable potentially associated with performance in both cognitive tasks and functioning was patient education. In this sample, participant educational level is confounded with patient status (patient M=13.0, control M=14.5, p<.01). Patient education was not controlled because the lower level of educational achievement observed in the patients was viewed as a consequence of the disorder given that schizophrenia is characterized by an initial onset in late adolescence or early adulthood and often interrupts secondary education (Meehl, 1969). To further elucidate the relationship between education and performance on the VRFCAT, we decided to use parental education as a proxy for the likely educational achievement of the participants in an attempt to exclude the influence of their illness. When controlling for parental education the group difference between patients and controls on the Total Completion Time remained statistically significant (Estimated marginal mean for Patients=821.19 secs vs. Controls=712.84 secs, F(1,39)=5.26, p=.027), but the difference in Total Number of Errors was attenuated (Estimated marginal mean for Patients=3.0 vs. Control M=1.0, F(1,39)=3.35, p=.075).

Correlations Between the Virtual Reality Functional Capacity Test (VRFCAT), Functional Capacity (UPSA), and Cognitive Functioning (MCCB and CAI)

All correlations in the patient group between the VRFCAT performance variables and the UPSA total score, the MCCB overall composite score, and the CAI Global Rater scores were statistically significant (p < .01) (Table 2). Within the patient group, the correlation between the VRFCAT total completion time and the MCCB overall composite score was strong (r=0.70, p= 0.01). In addition, despite the lack of group differences, the correlations of the VRFCAT Forced Progressions with cognition were all statistically significant.

Table 2.

Correlations Between the Virtual Reality Funtional Capacity Test (VRFCAT), Functional Capacity Assessment (UPSA), and Objective Cognitive (MCCB) and Interview-based Measures of Cognitive Functioning (CAI)

Virtual Reality Funcional Capacity Assessment Tool (VRFCAT)
 FunctionalCapacity (UPSA) Objective Cognition (MCCB) Cognitive Assessment Interview – Global (CAI)
Total Completion Time Total Number of Errors Total Forced Progressions Patient Informant Rater
VRFCAT
 Total Completion Time ------
 Total Number of Errors 0.653** ------
 Total Forced Progression 0.662** 0.834** ------
Functional Capacity (UPSA) −0.662** −0.530** −0.617** ------
Objective Cognition (MCCB) −0.703** −0.602** −0.658** 0.862** --------
Cognitive Assessment Interview - Global (CAI)
 Patient 0.388* 0.422** 0.352* −0.468** −0.660** -----
 Informant 0.503** 0.472** 0.414** −0.631** −0.651** 0.719** ------
 Rater 0.510** 0.505** 0.394* −0.634** −0.718** 0.841** 0.926** ------
Open in a new tab
*

Correlation is significant at the 0.05 level (2-tailed)

**

Correlation is significant at the 0.01 level (2-tailed)

Correlations Comparing Relationships between the Virtual Reality Functional Capacity Test (VRFCAT), Functional Capacity (UPSA), and Cognitive Measures (MCCB and CAI) with Functional Outcome (GFS-R, GFS-S, and the RFS)

In this set of correlational analyses, we compared the strength of the VRFCAT’s prediction of daily functioning with that of the standard approach to functional capacity, the UPSA (Table 3). We were also interested in determining the strength of the associations between the VRFCAT and functional outcome in a direct comparison with the MCCB and CAI. The target variables were the Global Functioning Scale: Role, the Global Functioning Scale: Social, and the separate domains assessed using Role Functioning Scale: 1) Work Functioning, 2) Independent Living, 3) Family Relationships, and 4) Social Functioning. We found the strength of the association between the VRFCAT Total Completion Time was as good as the UPSA’s association with levels of occupational and school functioning. Further, The VRFCAT and the UPSA were significantly correlated with social functioning, whereas the correlation of the MCCB and the CAI with social functioning did not reach significance. The independent living rating was significantly predicted by the UPSA and MCCB, but not the VRFCAT. Interestingly, none of the examined functional capacity or cognitive scores were significantly associated with family functioning.

Table 3.

Correlational Relationships between the Virtual Reality Functional Capacity Test (VRFCAT), Functional Capacity (UPSA), Cognitive Measures (MCCB and CAI), and Daily Functioning (GFS-R, GFS-S, and RFS)

Global Functioning Scale
 Role Functioning Scale
Role Functioning Social Functioning Work Functioning Independent Living Famly Relations Social Functioning
VRFCAT
 Total Time on Task −0.517** −0.433* −0.406* −0.187 −0.031 −0.464**
 Total Errors −0.171 −0.257 −0.152 −0.013 0.189 −0.187
 Total Forced Progressions −0.316 −0.432* −0.322 −0.165 −0.030 −0.416*
Functional Capacity (UPSA) 0.552** 0.487** 0.523** 0.547** 0.054 0.381*
Cognition (MCCB) 0.556** 0.358 0.470** 0.487** 0.072 0.288
Cognitive Assessment Interview (CAI)
 Patient −0.349 −0.175 −0.388 −0.182 0.011 −0.061
 Information −0.389* −0.311 −0.305 −0.214 0.060 −0.186
 Global Rater −0.452* −0.351 −0.364 −0.311 0.022 −0.233
Open in a new tab
*

Correlation is significant at the 0.05 level (2-tailed)

**

Correlation is significant at the 0.01 level (2-tailed)

Discussion

As hypothesized, the current study confirmed that the functional capacity (FC) performance deficit, as measured by the VRFCAT, was present in early course schizophrenia patients in that they performed more slowly and committed more errors than did control subjects. In addition, we found the VRFCAT was significantly correlated with an established paper and pencil version of functional capacity (UPSA), cognitive measures (MCCB and CAI), and functional outcome (GFS-R, GFS-S, RFS). The robust correlations between the VRFCAT and UPSA are considered an indicator of the VRFCAT’s validity as a functional capacity measure, even in the early course of schizophrenia. In addition, the VRFCAT showed construct validity in that we found significant correlations with objective (MCCB) and interview-based measures (CAI) of cognitive functioning. Interestingly, within the patient group, the correlation between the VRFCAT total time and the MCCB overall composite score (r=0.70, p= 0.01) was comparable in magnitude to the correlation often reported in the literature for the relationship between the UPSA and the MCCB (r=0.65). Further, the VRFCAT correlations with daily functioning were comparable in magnitude when compared to established measures such as the UPSA, MCCB, and the CAI in relationship to occupational and school functioning. The associations we found between VRFCAT and performance on the UPSA, MCCB, and the CAI supported previous reports by Keefe and colleagues (2016) in multi-episode schizophrenia patients of the VRFCAT’s validity as an assessment tool for functional capacity that is related to daily functioning. The VRFCAT could qualify as a possible co-primary measure in clinical trials evaluating treatments for cognitive impairment in schizophrenia.

Despite the VRFCAT group differences between the patients and controls on task completion time and number of errors, there was no statistically significant difference between groups for the Number of Forced Progressions. Yet, all three VRFCAT variables were indeed highly inter-correlated. On the VRFCAT, more errors and more forced progressions seems to have contributed to a longer time of completion. The VRFCAT is a complex task that requires a coordinated combination of multiple cognitive abilities to complete. Both errors and slower speed in being able to perform the functional capacity tasks might help account for poorer daily functioning. In fact, tests with time-based dependent variables have been found repeatedly to be strong correlates of composite neuropsychological performance (Keefe et al., 2006), performance on measures of functional capacity (McClure et al., 2007), and everyday outcomes (Harvey et al., 2009). Speed of processing has been shown to represent the largest deficit in patients compared to control subjects (Dickinson et al., 2007; Kern et al., 2011) and has demonstrated substantial importance for prediction of functioning level in schizophrenia (Bowie et al., 2008; Milev, 2005). However, error-based scores on measures of attention, working memory, and verbal and visual memory are also strongly correlated with functional capacity and everyday functioning (Green, 1996; Green et al., 2000). Thus, the multifaceted cognitive demands of the VRFCAT most likely contribute to its strong relationship to everyday functioning.

We extend previous findings in patients with an established illness in that these early course patients with schizophrenia showed virtual reality based functional capacity performance deficits when compared with controls. This is additional evidence the deficits that are present in more advanced stages of schizophrenia are not simply the result of illness chronicity or long-term exposure to antipsychotic medication. These deficits are perhaps developmental considering that they are present so early in the illness in young adult patients.

Virtual reality-based performance was correlated with cognitive functioning, suggesting that the VRFCAT might be sensitive to changes in cognition in young patients during the early course of schizophrenia. Our findings support the notion that the VRFCAT can be a viable co-primary outcome measure in clinical trials or treatment studies as has been suggested (Keefe et al., 2016). Given that the sample consisted of young people, the VRFCAT seemed appealing perhaps because of its computerized game-like quality. The examiners reported that the VRFCAT was simple to administer and noted a total administration time of about 30 minutes.

The VRFCAT was correlated with functional outcome measures that were previously used in treatment studies of cognitive impairment in schizophrenia. Virtual reality-based performance was highly correlated with several domains of functional outcome, suggesting that the VRFCAT might be sensitive to changes in functioning. Both the VRFCAT and the MCCB were correlated with work and school functioning. Interestingly, the VRFCAT and UPSA were more highly correlated with social functioning than was the MCCB and the CAI. This could be a reflection of the VRFCAT’s built in components requiring that the subject engage in community-based tasks such as taking public transportation and food shopping. Interestingly, none of our measures of functional capacity or cognitive functioning were significantly correlated with familial relationships indicating that family interactions might be driven by factors other than function capacity or cognitive capacities. This could be explained by a sense of togetherness among family members and their ill relative which suggests that family members learn to overlook the patient’s functional deficits.

As is the case with all research studies, this study has limitations. The first is the small sample size of controls. However, the findings fall in line with a priori hypotheses and prior findings on schizophrenia patients in more advanced stages of illness. Also, there was an imbalance in the gender distribution between the patients and the controls in that there were more females in the control group. These types of sex ratio imbalances are common in studies of schizophrenia patients that use controls, given the greater incidence rate of male participants in schizophrenia studies. We did not find any evidence indicating that the sex of the participants played a significant role in their performance on the VRFCAT, nor did Keefe et al 2016 with a larger sample. In addition, the reported correlations within our sample of schizophrenia patients are all cross-sectional in nature. Longitudinal research is needed to more directly establish that changes in the VRFCAT are related to changes in cognition and everyday functioning.

Acknowledgements

The findings from this study were presented in part at the 6th bi-annual meeting of the Schizophrenia International Research Society, Florence, Italy, April 4–8, 2018; Virtual Reality Assessment of Functional Capacity in Early Schizophrenia: Associations with Neurocognition, Functional Capacity Performance, and Daily Functioning. Joseph Ventura, Ph.D., Tamara Welikson, M.A., Kenneth L. Subotnik, Ph.D., Arielle Ered, M.A., Gerhard S. Hellemann, Ph.D., Richard S.E. Keefe, Ph.D., and Keith H. Nuechterlein, Ph.D.

Data from this study were used in partial fulfillment for Tamara Welikson’s doctoral dissertation awarded by Alliant University, Alhambra, 2018.

This research was supported in part by National Institute of Mental Health Grants R01 MH37705, R34 MH102529, and P50 MH066286 to Keith H. Nuechterlein, Ph.D.

Footnotes

Conflict of Interest Statement

Joseph Ventura, Ph.D. has received funding from Brain Plasticity, Inc., Genentech, Inc., and Janssen Scientific Affairs, LLC, and has served as a consultant to Boehringer-Ingelheim, GmbH, and Brain Plasticity, Inc.

Kenneth L. Subotnik, Ph.D. has served as a consultant to Alkermes, Inc, Teva Pharmaceuticals, and Medincell, Inc, and has been on the speaker’s bureaus for Janssen Canada and Otsuka America Pharmaceutical, Inc.

Keith Nuechterlein, Ph.D. has been a consultant to Astellas, Genentech, Janssen, Medincell, Otsuka, Takeda, and Teva, and he has received unrelated research, grants from Genentech, Janssen Scientific Affairs and Posit Science.

Richard Keefe, Ph.D. is currently or in the past 3 years has received honoraria, served as a consultant, speaker, or advisory board member for Abbvie, Acadia, Aeglea, Akebia, Akili, Alkermes, Allergan, ArmaGen, Astellas, Avanir, AviNeuro/ChemRar, Axovant, Biogen, Boehringer-Ingelheim, Cerecor, CoMentis, Critical Path Institute, FORUM, Gammon Howard & Zeszotarski, Global Medical Education (GME), GW Pharmaceuticals, Intracellular Therapeutics, Janssen, Kempharm, Lundbeck, Lysogene, MedScape, Mentis Cura, Merck, Merrakris Therapetics, Minerva Neurosciences Inc., Mitsubishi, Montana State University, Monteris, Moscow Research Institute of Psychiatry, Neuralstem, Neuronix, Novartis, NY State Office of Mental Health, Orygen, Otsuka, Paradigm Testing, Percept Solutions, Pfizer, Pharm-Olam, Regenix Bio, Reviva, Roche, Sangamo, Sanofi, SOBI, Six Degrees Medical, Sunovion, Takeda, Targacept, Teague Rotenstreich Stanaland Fox & Holt, Thrombosis Research Institute, University of Moscow, University of Southern California, University of Texas Southwest Medical Center, WebMD, and Wilson Therapeutics. Dr. Keefe has currently or in the past 3 years received research funding from the National Institute of Mental Health.

Dr. Keefe also receives royalties from versions of the BAC testing battery, the MATRICS Battery (BACS Symbol Coding), and the Virtual Reality Functional Capacity Assessment Tool (VRFCAT). He is also a shareholder in NeuroCog Trials, Inc. and Sengenix.

Tamara Welikson, Arielle Ered, and Gerhard Hellemann each report no conflict of interest.

References

  1. Becattini-Oliveira AC, de Farias Dutra D, de Oliveira Campos BS, de Araujo VC, Charchat-Fichman H, 2018. A systematic review of a functional assessment Tool: UCSD Performance-based skill assessment (UPSA). Psychiatry research. [DOI] [PubMed] [Google Scholar]
  2. Bowie CR, Harvey PD, 2008. Communication abnormalities predict functional outcomes in chronic schizophrenia: differential associations with social and adaptive functions. Schizophr Res 103(1–3), 240–247. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bowie CR, Leung WW, Reichenberg A, McClure MM, Patterson TL, Heaton RK, Harvey PD, 2008. Predicting schizophrenia patients’ real-world behavior with specific neuropsychological and functional capacity measures. Biological Psychiatry 63(5), 505–511. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bowie CR, Reichenberg A, Patterson TL, Heaton RK, Harvey PD, 2006. Determinants of real-world functional performance in schizophrenia subjects: Correlations with cognition, functional capacity, and symptoms. American Journal of Psychiatry 163(3), 418–425. [DOI] [PubMed] [Google Scholar]
  5. Cornblatt BA, Auther AM, Niendam T, Smith CW, Zinberg J, Bearden CE, Cannon TD, 2007. Preliminary findings for two new measures of social and role functioning in the prodromal phase of schizophrenia. Schizophr Bull 33(3), 688–702. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Dickinson D, Ramsey ME, Gold JM, 2007. Overlooking the obvious: a meta-analytic comparison of digit symbol coding tasks and other cognitive measures in schizophrenia. Arch Gen Psychiatry 64(5), 532–542. [DOI] [PubMed] [Google Scholar]
  7. Goodman SH, Sewell DR, Cooley EL, 1993. Assessing levels of adaptive functioning: The role functioning scale. Community Mental Health Journal 29, 119–131. [DOI] [PubMed] [Google Scholar]
  8. Green MF, 1996. What are the functional consequences of neurocognitive deficits in schizophrenia? Am J Psychiatry 153(3), 321–330. [DOI] [PubMed] [Google Scholar]
  9. Green MF, Kern RS, Braff DL, Mintz J, 2000. Neurocognitive deficits and functional outcome in schizophrenia: Are we measuring the “right stuff”? Schizophrenia Bulletin 26(1), 119–136. [DOI] [PubMed] [Google Scholar]
  10. Green MF, Schooler NR, Kern RS, Frese FJ, Granberry W, Harvey PD, Karson CN, Peters N, Stewart M, Seidman LJ, Sonnenberg J, Stone WS, Walling D, Stover E, Marder SR, 2011. Evaluation of functionally meaningful measures for clinical trials of cognition enhancement in schizophrenia. American Journal of Psychiatry 168(4), 400–407. [DOI] [PubMed] [Google Scholar]
  11. Green RS, Gracely EJ, 1987. Selecting a rating scale for evaluating services to the chronically mentally ill. Community Mental Health Journal 23, 91–102. [DOI] [PubMed] [Google Scholar]
  12. Harvey PD, Keefe RS, Patterson TL, Heaton RK, Bowie CR, 2009. Abbreviated neuropsychological assessment in schizophrenia: prediction of different aspects of outcome. J Clin Exp Neuropsychol 31(4), 462–471. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Keefe RS, Bilder RM, Harvey PD, Davis SM, Palmer BW, Gold JM, Meltzer HY, Green MF, Miller del D, Canive JM, Adler LW, Manschreck TC, Swartz M, Rosenheck R, Perkins DO, Walker TM, Stroup TS, McEvoy JP, Lieberman JA, 2006. Baseline neurocognitive deficits in the CATIE schizophrenia trial. Neuropsychopharmacology 31(9), 2033–2046. [DOI] [PubMed] [Google Scholar]
  14. Keefe RS, Davis VG, Atkins AS, Vaughan A, Patterson T, Narasimhan M, Harvey PD, 2016. Validation of a computerized test of functional capacity. Schizophrenia research 175(1–3), 90–96. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Keefe RSE, Goldberg TE, Harvey PD, Gold JM, Poe M, L C, 2004. The Brief Assessment of Cognition in Schizophrenia: Reliability, sensitivity, and comparison with a standard neurocognitive battery. Schizophrenia Research 68(2–3), 283–297. [DOI] [PubMed] [Google Scholar]
  16. Kern RS, Gold JM, Dickinson D, Green MF, Nuechterlein KH, Baade LE, Keefe RS, Mesholam-Gately RI, Seidman LJ, Lee C, 2011. The MCCB impairment profile for schizophrenia outpatients: results from the MATRICS psychometric and standardization study. Schizophrenia research 126(1–3), 124–131. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Laughren T, 2001. The scientific and ethical basis for placebo-controlled trials in depression and schizophrenia: an FDA perspective. European Psychiatry 16(7), 418–423. [DOI] [PubMed] [Google Scholar]
  18. McClure MM, Bowie CR, Patterson TL, Heaton RK, Weaver C, Anderson H, Harvey PD, 2007. Correlations of functional capacity and neuropsychological performance in older patients with schizophrenia: Evidence for specificity of relationships? Schizophrenia Research 89(1–3), 330–338. [DOI] [PubMed] [Google Scholar]
  19. Meehl PE, 1969. Nuisance variables and the ex post facto design. Department of Psychiatry, University of Minnesota. [Google Scholar]
  20. Milev P, 2005. Predictive Values of Neurocognition and Negative Symptoms on Functional Outcome in Schizophrenia: A Longitudinal First-Episode Study With 7-Year Follow-Up. American Journal of Psychiatry 162(3), 495–506. [DOI] [PubMed] [Google Scholar]
  21. Narvaez JM, Twamley EW, McKibbin CL, Heaton RK, Patterson TL, 2008. Subjective and objective quality of life in schizophrenia. Schizophrenia research 98(1–3), 201–208. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Niendam TA, Bearden CE, Johnson JK, Cannon TD, 2006. Global Functioning Scale: Role (GFS: Role). University of California, Los Angeles, Los Angeles. [Google Scholar]
  23. Nuechterlein KH, Barch DM, Gold JM, Goldberg TE, Green MF, Heaton RK, 2004. Identification of separable cognitive factors in schizophrenia. Schizophrenia Research 72(1), 29–39. [DOI] [PubMed] [Google Scholar]
  24. Nuechterlein KH, Green MF, Kern RS, Baade LE, Barch DM, Cohen JD, Essock S, Fenton WS, Frese FJ 3rd, Gold JM, Goldberg T, Heaton RK, Keefe RS, Kraemer H, Mesholam-Gately R, Seidman LJ, Stover E, Weinberger DR, Young AS, Zalcman S, Marder SR, 2008. The MATRICS Consensus Cognitive Battery, part 1: Test selection, reliability, and validity. American Journal of Psychiatry 165(2), 203–213. [DOI] [PubMed] [Google Scholar]
  25. Nuechterlein KH, Subotnik KL, Green MF, Ventura J, Asarnow RF, Gitlin MJ, Yee CM, Gretchen-Doorly D, Mintz J, 2011. Neurocognitive predictors of work outcome in recent-onset schizophrenia. Schizophr Bull 37 Suppl 2, S33–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Nuechterlein KH, Ventura J, Subotnik KL, Hayata JN, Medalia A, Bell MD, 2014. Developing a Cognitive Training Strategy for First-Episode Schizophrenia: Integrating Bottom-Up and Top-Down Approaches. Am J Psychiatr Rehabil 17(3), 225–253. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Patterson TL, Goldman S, McKibbin CL, Hughs T, Jeste DV, 2001. UCSD Performance-Based Skills Assessment: Development of a new measure of everyday functioning for severely mentally ill adults. Schizophrenia Bulletin 27(2), 235–245. [DOI] [PubMed] [Google Scholar]
  28. Patterson TL, Lacro J, McKibbon CL, Moscona S, Hughs T, Jeste DV, 2002. Medication management ability assessment: Results from a performance-based measure in older outpatients with schizophrenia. Journal of Clinical Psychopharmacology 22(1), 11–19. [DOI] [PubMed] [Google Scholar]
  29. Ruse SA, Harvey PD, Davis VG, Atkins AS, Fox KH, Keefe RS, 2014. Virtual reality functional capacity assessment in schizophrenia: Preliminary data regarding feasibility and correlations with cognitive and functional capacity performance. Schizophrenia Research: Cognition 1(1), e21–e26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Sabbag S, Twamley EM, Vella L, Heaton RK, Patterson TL, Harvey PD, 2011. Assessing everyday functioning in schizophrenia: not all informants seem equally informative. Schizophrenia research 131(1–3), 250–255. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Strassnig MT, Raykov T, O’Gorman C, Bowie CR, Sabbag S, Durand D, Patterson TL, Pinkham A, Penn DL, Harvey PD, 2015. Determinants of different aspects of everyday outcome in schizophrenia: the roles of negative symptoms, cognition, and functional capacity. Schizophrenia research 165(1), 76–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Ventura J, Reise SP, Keefe RSE, Baade LE, Gold JM, Green MF, Kern RS, Mesholam-Gately R, Nuechterlein KH, Seidman LJ, 2010. The Cognitive Assessment Interview (CAI): Development and validation of an empirically derived, brief interview-based measure of cognition. Schizophrenia Research 121(1–3), 24–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Ventura J, Subotnik KL, Ered A, Hellemann GS, Nuechterlein KH, 2016. Cognitive Assessment Interview (CAI): Validity as a co-primary measure of cognition across phases of schizophrenia. Schizophr Res 172(1–3), 137–142. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Vesterager L, Christensen TØ, Olsen BB, Krarup G, Melau M, Forchhammer HB, Nordentoft M, 2012. Cognitive and clinical predictors of functional capacity in patients with first episode schizophrenia. Schizophrenia research 141(2–3), 251–256. [DOI] [PubMed] [Google Scholar]

RESOURCES