Virtual reality assessment of functional capacity in people with Schizophrenia: Associations with reduced emotional experience and prediction of functional outcomes

Abstract

Virtual Reality (VR) approaches have had considerable success in measurement of functional capacity. However, it is not clear if factors other than cognitive impairment influence performance on VR measures. Many people with schizophrenia have significant negative symptoms and they could reduce engagement in assessment. 158 patients with schizophrenia performed the VRFCAT, were tested with the MCCB, were rated with the PANSS, and were rated on everyday functioning. Scores for reduced emotional experience and reduced expression were derived. Reduced emotional experience, but not reduced expression, was correlated with socially relevant VRFCAT subtasks and real-world social functioning. Performance on the socially relevant subtasks, but not the solitary subtasks, shared variance with work outcomes. MCCB performance was associated with both subdomains, but socially relevant subtasks shared more variance. Patients with higher reduced emotional experience validly engaged in socially relevant VR simulations, as indexed by correlations with outcome measures. These patients had poorer performance on socially relevant tasks than on solitary tasks. The differential validity of solitary vs. socially relevant simulations was supported by differences in correlates, suggesting that assessments with a focus on performance of simulated socially relevant tasks could be developed.

1. Introduction

Schizophrenia is associated with notable disability across multiple functional domains. The origin of disability comes from impairments in cognitive performance (Green et al., 2000), negative symptoms, particularly avolition-anhedonia (Strassnig et al., 2015; Strauss et al., 2013; Harvey et al., 2017), and challenges in performance of everyday functional skills (Bowie et al., 2006). Treatments for schizophrenia include antipsychotic medications, which have been shown to be effective at managing and reducing positive symptoms (Leucht et al., 2013). Yet, refractory symptoms, cognitive deficits, social cognitive deficits, and negative symptoms often persist (Dodell-Feder et al., 2015; Abi-Dargham, 2014). Additionally, some data suggests that avolition-anhedonia (amotivation; Foussias and Remington, 2010; Kirkpatrick, 2006) is more strongly linked to social impairments than to other outcomes, there is likely a broad impact of reduced emotional experience across different functional domains (Ventura et al., 2015).

Consistent with this idea, Illerena et al. (2018) reported that reduced emotional experience, but not reduced expression was associated with employment outcomes. In specific, getting a job keeping a job, and hours worked were all correlated with reduced emotional experience symptoms. However, when age was considered, these effects became nonsignificant, suggesting perhaps that the influence of negative symptoms was moderated by other variables. In a related study of employment outcomes, Kern et al. (2018) reported that a combination therapy of errorless learning and supported employment interventions led to better work outcomes than supported employment alone. Interesting, social skills were a substantial moderator of these outcomes, accounting for an additional 18% of the variance in successful work outcomes. As negative symptoms of reduced emotional experience have been found to interact with both social skills and social cognition (Kalin et al., 2015), consideration of factors associated social functioning appears very important when attempting to understand the determinants of work success in people with severe mental illness.

Impairments in functional capacity, including skills required for employment, everyday activities, and social interactions, have been reported to be proximal to impairments in everyday functioning across neuropsychiatric conditions (Bowie et al., 2010). Cognitive and social cognitive impairments have been reported to predict challenges in impairments in functional and social skills which in turn predict impairments in functional outcomes (Strassnig et al., 2015; Kalin et al., 2015). Some recent research has suggested that variations in the severity of avolition-anhedonia symptoms may have implications for prediction of social outcomes. Both lower and higher levels of reduced emotional experience were recently found to be correlated with social functioning deficits (Strassnig et al., 2018), but in patients with higher levels of symptom severity, social cognitive performance was not a predictor of social outcomes (Harvey et al., 2018). In that study, for individuals with lower levels of reduced emotional experience there were 4 different performance-based social cognitive tests significantly associated with informant-rated social outcomes, in contrast to none in the individuals with greater symptom severity.

One aspect of reduced emotional experience that has not been studied in detail is the impact on performance-based assessments of social and functional skills of this symptom complex. It is possible that the association between greater reductions in emotional experience and reduced impact of social cognitive deficits is because these symptoms directly interfere with the ability to perform specific socially relevant tasks. It is possible that individuals with notable reductions in emotional experience, defined in large part by reduced social interaction and emotional engagement, may be overly sensitive to the demands of socially focused assessments. An important question, since technology is increasingly being used to assess social and functional skills (Keefe et al., 2016; Czaja et al., 2017), is whether reduced emotional experience symptoms could be triggered by VR simulations of social interactions, which could lead to challenges in performing these tasks. Virtual reality (VR) technology, with its ability to simulate multifaceted, real situations and contexts, allows for the investigation of human behavior in well controlled designs in the laboratory. VR strongly depend on the adequate selection of unambiguous perceptual cues to trigger emotions. Additionally, emotional experiences are related to presence (Botella et al., 2009; Schubert et al., 2001), which depicts the user’s sense of being in a VR environment. Virtual environments for functional or cognitive assessment engage subjects by allowing them to be involved in a task while being less focused on the fact that they are being tested. Relevant functional tasks, such as navigating a virtual community, or shopping and paying for groceries in a virtual supermarket, can be delivered with improved ecological validity compared to traditional questionnaires or performance‐based assessments.

In this study, we use the data from a previously published study assessing the performance of a VR assessment of the ability to perform everyday functional skills, the Virtual Reality Functional Capacity Assessment Test (VRFCAT; Keefe et al., 2016) in individuals with schizophrenia. The VRFCAT is an immersive VR simulation of a series of everyday tasks associated with meal preparation, travel and transit, shopping, and financial skills. The task has been shown to be broadly sensitive to impairments in functional capacity including diagnostic differences between schizophrenia patients and healthy controls (Ruse et al., 2014; Keefe et al., 2016), correlations with cognitive test performance and paper and pencil functional capacity measures (Ruse et al., 2014; Keefe et al., 2016), sensitivity to age effects in healthy people (Atkins et al., 2015), and sensitivity to effects of subjective cognitive complaints in healthy people (Atkins et al., 2018). There are 12 different objectives in the task, of which 5 are performed while home alone and the other 7 are performed out of the participant’s virtual residence. Thus, there are 7 tasks that are performed in an environment where there are either required social interactions (e.g., paying the driver for the bus, paying a cashier) and or implied potential for social interactions (e.g., walking around in a supermarket; waiting for a bus). As a result, comparison of performance across solitary vs. social situations and its relationship with the socially-relevant symptoms of reduced emotional experience was directly examined.

Our hypothesis was that negative symptoms of reduced emotional experience would have a negative relationship with performance on socially engaged VR tasks, in contrast to solitary tasks. We also hypothesized, based on previous studies of the differential correlations between different elements of negative symptoms and social outcomes, that reduced emotional expression would be less associated than reduced experience with performance on both types of tasks. Further, we expected that performance on socially involved tasks would be more strongly correlated with elements of real-world social functioning, rated independently by high contact informants, than solitary tasks. Significant correlations between VRFCAT performance and reduced emotional experience could also mean that patients with reduced emotional experience are able to engage in a VR task in a meaningful manner.

2. Methods

The results of this study were published previously (Keefe et al., 2016), although analyses of negative symptoms subtypes were not part of that publication, nor were the analyses of VRFCAT performance as function of the level of social demands of the simulation items. As a result, the methods are presented in an abbreviated version. Further, the analyses of the correlations of the total VRFCAT scores, cognitive test performance, informant ratings of cognitive performance with the Schizophrenia Cognition Rating Scale (SCoRS; Keefe et al., 2006), and the UCSD performance-based skills assessment-Brief Version (UPSA-B; Mausbach et al., 2007) were previously published in detail and are not repeated in these analyses. Herein we simply examine the correlation between cognitive performance and the VRFCAT sub-domain scores.

2.1. Participants

Schizophrenia participants were recruited at three research sites: (1) The University of South Carolina under the supervision of Dr. Meera Narasimhan; (2) The University of Miami Miller School of Medicine under the supervision of Dr. Philip Harvey; and (3) The University of California, San Diego School of Medicine under the supervision of Dr. Thomas Patterson.

Patients met criteria for DSM-IV TR schizophrenia, any subtype. All patients completed a structured diagnostic interview, administered by a trained interviewer: Mini International Neuropsychiatric Interview, 6th Edition (Sheehan et al., 1998). Patients with Positive and Negative Syndrome Scale (PANSS, Kay et al., 1987) symptom severity scores greater than 5 (“moderately severe”) on either item P1 or P3 (delusions or hallucinatory behavior) were excluded from the study, in line with the standards for cognitive enhancement clinical trials from the MA-TRICS initiative (Buchanan et al., 2005; Buchanan et al, 2011). Patients were also screened for their ability to engage in testing. Those who were uncooperative, suffered from extreme cognitive impairment, had another DSM-IV diagnosis that would exclude the diagnosis of schizophrenia, or had severely limited eyesight were excluded. Participants who participated in studies of cognition with any of the same measures within the last 12 months were not included. None of the participants in the previous feasibility study of the VRFCAT (Ruse et al., 2014) participated. Other exclusionary criteria included inability to provide personal informed consent, history of brain trauma, documented neurologic disorder, medical conditions interfering with daily functioning, and current or recent substance abuse. Approval from the Institutional Review Board (IRB) was secured at each site and at the Sponsor site and all participants provided signed informed consent.

2.2. VRFCAT description

The VRFCAT measures four different functional abilities: checking for the availability of items to complete a recipe, taking a bus, shopping in a store, and managing currency. All participants received a brief tutorial, which included sample items similar to those from the test and practice in using the mouse and computer. There were 12 different objectives, presented in Table 1. For each objective the dependent variables were accuracy of performance and time to completion. For all objectives, participants who were unable to complete the objective within a pre-specified time were given a time to completion score of 300 s for that objective and automatically progressed to the next objective. Six different forms of the VRFCAT were developed and tested in this study, with forms randomized across participants. In line with the results of the validation study, these forms were combined and not examined separately in this study.

Table 1.

VRFCAT objectives.

Objective	Description
1.	Pick up the recipe.
2.	Search for Ingredients
3.	Cross off correct ingredients and pick up the bus schedule
4.	Pick up the billfold
5.	Exit the apartment
6.	Get on the bus to the grocery store
7.	Pay for the bus
8.	Select an aisle
9.	Shop for groceries
10.	Pay for groceries
11.	Get on the bus to go home
12.	Pay for the bus

Note. Objectives 1–5 are Performed alone at home and 7–12 are performed away from home.

2.3. MATRICS Consensus Cognitive Battery (MCCB)

The MCCB (Nuechterlein et al., 2008) measures seven separable cognitive domains: speed of processing; attention/vigilance; working memory (verbal and nonverbal); verbal learning; visual learning; reasoning and problem solving; and social cognition. Herein we excluded social cognition and created a neurocognitive composite score. Administration of the MCCB requires about 75–90 min. The subtests were administered in the standard order. The MCCB scoring program yields seven domain scores and a composite score, which are standardized to the same T-score measurement scale with a mean of 50 and an SD of 10 (Kern et al, 2008).

2.4. Real-world functional outcomes

The rating scale employed was the Specific Levels of Functioning (SLOF; Schneider and Struening, 1983). The SLOF version used was a 31-item informant-rated assessment of behavior and functioning which was abbreviated to assess only the following three domains: Interpersonal Functioning, Everyday Activities, and Vocational Functioning. The informant was selected based on knowing the patient very well, with a high contact clinician or caregiver given priority informant (family member, professional caregiver, social worker, or case manager). The dependent variable for the statistical analyses was total score for each of the three different subscales, selected because there were no missing ratings.

2.4.1. Time distribution

We asked participants how much time they typically home during the daytime hours. As the patient sample was 85% unemployed, time spent at work activities was not examined

2.5. Symptoms assessment

Severity of symptoms was evaluated with the Positive and Negative Syndrome Scale (PANSS), which was administered in its entirety by trained raters. These raters had extensive experience in other studies of patients with schizophrenia. The PANSS consists of 30 items with 3 subscales: 7 items were the positive symptoms scale (P1–P7), 7 items were the negative symptoms scale (N1–N7), and 16 items were general psychopathology symptoms scale (G1–G16). Each item was scored on a 7-point Likert scale ranging from 1 to 7. Thus, the range of the positive and the negative sub-scales were 7 to 49 while the range of the general psychopathology scale was 16 to 112 (Kay et al., 1987).

2.5.1. Negative symptom models

A two-factor model of expression and experience was created and tested in several samples in a study by Khan et al. (2017). The model was also examined for its link to functional outcomes (Harvey et al., 2017; Strassnig et al., 2018). The items in the PANSS Reduced Emotional Experience factor are: Emotional Withdrawal (N2), Passive/Apathetic Social Withdrawal (N4) and Active Social Avoidance (G16). The items in the PANSS Reduced Emotional Expression factor are: Blunted Affect (N1), Poor Rapport (N3), Lack of Spontaneity and Flow of Conversation (N6), and Motor Retardation (G7).

2.6. Data analyses

As total time to completion was the best performing variable in the previous validation study of the VRFCAT, we used that variable as our primary correlational variable. We created scores for at home (Solitary) and away from home (Socially Relevant) variables by summing completion times in each domain. We also considered average time to completion for each task demand and found the same correlations with the other variables, leading us to use the total scores for the home and away task demands.

We computed Pearson Correlation Coefficients (r) between the 2 VRFCAT scores (Solitary; Socially Relevant), the two negative symptoms subscales (Reduced Emotional Expression; Reduced Emotional Experience), the MCCB composite t-score, and the three SLOF variables (Interpersonal Functioning; Everyday Activities; Vocational Functioning). We then used the two negative symptom variables in a stepwise regression model to predict activities performed at home vs. those performed away. Finally, we predicted the 3 SLOF outcome variables in regressions, using the two domains of negative symptoms and the two elements of VRFCAT performance.

3. Results

Descriptive information on the patients was previously published and is contained in supplemental Table 1. Performance on the two VRFCAT domains, the severity scores on negative symptom subdomains, SLOF scores, MCCB composite scores, and time spent at home are presented in Table 2. In terms of performance on the VRFCAT, the relative time to complete the tasks in each area was generally consistent with the number of tasks (5 vs. 7) for home and away activities. For negative symptoms, there was a wide range of scores in both domains, with some patients having none of the symptoms (score of 3 or 4) and the maximum being an average item score of severe (5). For the SLOF, there was also a range of scores obtained. MCCB total scores, on average, were performed 2.0 SD worse than normative standards.

Table 2.

Performance on VRFCAT Variables, Negative Symptom Severity, Cognitive Performance and SLOF Informant Rated Functioning.

	(n = 158)
	M	SD	Metric	Range
Time spent at home	6.2	11.6	Hours	0–8
Negative symptoms
Reduced emotional experience	8.6	3.6	Score	3–17
Reduced emotional expression	8.9	3.7	Score	4–19
VRFCAT variables
Activities at home	357.1	106.7	Seconds	194–1058
Activities away from home	672.8	221.7	Seconds	378–1557
SLOF subscales
Interpersonal	22.7	5.5	Score	11–35
Everyday activities	48.8	5.7	Score	32–63
Work	20.6	4.6	Score	8–30
MCCB neurocognitive composite	29.82	12.50	t-score	1–63

3.1. Correlational analyses

Pearson correlations between Negative Symptoms, VRFCAT, and SLOF variables are in Table 3. Cognitive performance correlated with better VRFCAT performance on both domains, more time spent at home, lower negative symptoms of both types, and SLOF everyday activities and work. Time spent at home correlated with better performance on VRFCAT activities performed at home, and less so with activities done away from home. Time spent at home was also correlated with higher scores on work abilities. Reduced emotional experience was correlated with reduced emotional expression and predicted poorer scores on VRFCAT activities performed away from home, but not at home. Consistent with earlier work, reduced emotional experience was negatively correlated with interpersonal functioning, but not associated with activities or work. Reduced emotional expression was correlated only with lower scores on everyday activities. The only correlate of VRFCAT activities performed at home, other than time at home, was performance on away from home activities. Poorer performance on VRFCAT away from home activities was negative associated with more severe symptoms of reduced emotional experience and poorer scores on work skills. As would be expected, the SLOF subscales were intercorrelated.

Table 3.

Correlations between VRFCAT Variables, Negative Symptom Severity, and SLOF Informant Rated Functioning.

(n = 158)
	1	2	3	4	5	6	7	8	9
1. Time at home	1.0	−0.11	.13	.26**	.32***	.04	.14	−0.21**	.23**
2. Reduced experience		1.0	.50***	.04	.18*	−0.37***	−0.12	−0.15	−0.32***
3. Reduced expression			1.0	.12	.04	−0.03	−0.18*	−0.02	−0.34***
4. VRFCAT solitary tasks				1.0	.70***	.03	−0.05	−0.11	−0.37***
5. VRFCAT socially relevant tasks					1.0	−0.07	−0.13	−0.23**	−0.43***
6. SLOF interpersonal						1.0	.32***	.47***	.14
7. SLOF activities							1.0	.51***	.30***
8. SLOF work								1.0	.34***
9. MCCB composite score									1.0

^*p < .05

^**p < .01

^***p < .001

Note. Higher scores on Reduced Expression and experience reflect more impairment. VRFCAT scores in expressed in terms of time to completion: higher scores reflect poorer performance; SLOF scores: Higher scores reflect better functioning

3.2. Regression analyses

There were four different patterns of correlations observed. First, cognitive performance was correlated with both VRFCAT Solitary and VRFCAT Socially Relevant activities, but correlation with socially relevant activities was higher (r = −.43). Second, work skills were negatively correlated with time spent at home (r = −.21) and with longer time required to complete VRFCAT socially relevant activities (r = −.23). Third, both sets of VRFCAT activities were correlated with time spent at home, but to a different extent, (r = .26 and .32 respectively), with more time spent at home correlating with poorer performance on both types of activities. Finally, reduced emotional experience appeared to be more related to VRFCAT socially relevant activities (r = .18).

In a first regression analysis, we entered VRFCAT solitary and socially relevant activities as well as time spent at home in a stepwise regression analysis predicting cognitive performance. The overall analysis was significant, F(1155) = 33.8, p < .001. VRFCAT socially relevant activities entered the equation first, accounting for 18% of the variance, t = 5.82, p < .001. Solitary activities did not add any significant variance to this prediction, t = 1.62, p = .110 and neither did time spent at home, t = 1.68, p = .095.

In the second regression analysis, we entered time spent at home, and VRFCAT solitary and socially relevant activities in a stepwise regression analysis predicting work skills. The overall analysis was significant, F(2156) = 6.30, p = .003. VRFCAT socially relevant activities the equation first, accounting for 5% of the variance, t = −2.91, p < .004, and time spent at home entered next, accounting for 3% of incremental variance, t = 1.98, p = .049. Solitary did not add any significant variance to this prediction, t = .97, p = .340.

In the third regression analysis, we predicted time spent at home with VRFCAT performance on socially relevant and solitary activities, using a similar stepwise regression model. The overall regression analysis was significant, F(1157) = 17.72, p < .001. Poorer performance on socially relevant activities entered the equation first, accounting for 10% variance, t = −4.23, p < .001, while VRFCAT solitary activities did not enter the equation, t = −.63, p = .520.

In the final regression analysis, we predicted VRFCAT socially relevant activities with reduced emotional experience and reduced emotional expression. The overall analysis was significant, F(1157) = 5.01, p = .027. Reduced emotional experience accounted for 4% of the variance in VRFCAT performance, t = 2.23, p = .027, but reduced expression did not enter the equation, t =−.66, p=.520.

4. Discussion

In this study, we found that individuals with schizophrenia manifested a relationship between symptoms of reduced emotional experience and several different functional outcomes. Specifically, VRFCAT performance was poorer on socially relevant tasks in patients with greater severity of reduced emotional experience. Regression analyses suggested that, in contrast, reduced emotional expression contributed no unique variance to the prediction of socially relevant VRFCAT activities. Thus, there is not a general impact of negative symptom severity, but rather a specific adverse impact on the performance of socially relevant RW tasks on the part of patients with reductions in their emotional experience. Replicating previous findings in non-overlapping samples (Harvey et al., 2017, 2018; Strassnig et al., 2018), reduced emotional experience defined by three items on the PANSS was also correlated with poorer real-world social functioning. Cognitive performance was associated with performance of both VRFCAT domains, but to a different extent.

These data suggest that patients with reductions in their emotional experience can engage in VR delivered assessments depicting social interactions and that their performance is associated with the severity of these symptoms. Evidence for the validity of the distinction between socially relevant and solitary VR tasks also comes from the differences in correlations with work performance, an activity performed outside the home which in most cases would also require social interactions, as well as differences in correlations with time spent at home. Better performance on VRFCAT socially relevant activities, but not VRFCAT solitary activities, contributed variance to ratings of better work ability. In contrast, time spent at home was associated with greater disadvantages in performance of social relevant activities, although both types of activities were performed more poorly by patients who reported spending more time at home and, presumably, generally alone. Finally, performance on VRFCAT socially relevant activities accounted for significant variance in cognitive performance while solitary VRFCAT activities did not.

There are some limitations in these data. The VRFCAT was not designed to assess solitary and socially relevant activities with equal numbers of items and there are more assessments of outside activities. We already demonstrated that cognitive performance and informant ratings on tan interview based measure of functional capacity, The Schizophrenia Cognition Rating Scale (SCoRS; Keefe et al., 2006) were substantially correlated with work and everyday activities in this sample (Keefe et al., 2016) and did not repeat those analyses here. The VRFCAT socially relevant activities variable, because of its overlap with cognitive performance, would not account for any variance in work outcomes if a regression analysis was to be performed considering cognition. The VRFCAT variables are highly correlated with each other and using regression analysis likely underestimates the independent contributions to other outcomes. Social outcomes are also likely predicted by social cognitive functioning, particularly in patients with lower levels of reduced emotional experience. Although the VRFCAT subdomains are variable in the extent to which their tasks are socially relevant, they are not formal assessments of social cognitive ability.

It should be noted that the correlations between work outcomes and the predictors, including VRFCAT subscales, cognitive performance, and reduced emotional expression, are all small in size, albeit independently contributing. This is consistent with previous studies predicting SLOF subscales using multiple predictors, wherein less than 50% of the total variance is accounted for and many predictors have minimal correlations with the outcomes (e.g., Bowie et al., 2008; Galderisi et al., 2014) It seems as if a complex real-world outcome such as work is likely to have multiple contributing factors, both those coming from the individuals (abilities, motivation) as well as those originating in the environment, such as opportunities and disability compensation. It is quite likely that the poor employment outcomes are not caused by single individual-level factors, but rather by a combination of a number of individual influences as suggested by these results.

In summary, these analyses suggest that social and solitary VR activities have different correlates in functional domains and are differentially related to negative symptoms. The assessment of functional capacity is vital in establishing an understanding of real-world functioning in patients with cognitive impairment in schizophrenia. Patients with reduced emotional experience not only can engage and participate in the VR activity, but manifest performance that suggests challenges in performing socially relevant VR tasks. Solitary tasks are not as strongly correlated with reduced emotional experience and are better performed by individuals who spend more time at home. These results suggest that there are multiple influences on performance-based VR tasks, including both specific symptom subdomains as well as cognitive performance, much like the multiple influences on real world outcomes. Using these tasks to assess people with schizophrenia appears to provide real-world relevant information as well as to elucidate the impact of socially relevant negative symptoms on simulations of socially demanding versus solitary tasks. These findings indicate that symptoms that influence virtual social interactions are similar to symptoms that affect community social interactions. Findings also demonstrate enhanced ecological validity for the VRFCAT to simulate social interactions while providing standardized visual and auditory experiences during testing. Knowledge of these factors has significant clinical implications and could aid in treatment and prevention strategies in schizophrenia.