Skip to main content
Schizophrenia Bulletin logoLink to Schizophrenia Bulletin
. 2020 Jul 2;47(1):108–117. doi: 10.1093/schbul/sbaa085

Online Social Cognition Training in Schizophrenia: A Double-Blind, Randomized, Controlled Multi-Site Clinical Trial

Mor Nahum 1,2,, Hyunkyu Lee 2, Melissa Fisher 3, Michael F Green 4,5, Christine I Hooker 6, Joseph Ventura 5, Joshua T Jordan 7, Annika Rose 2, Sarah-Jane Kim 2, Kristen M Haut 6, Michael M Merzenich 2, Sophia Vinogradov 3
PMCID: PMC7825077  PMID: 32614046

Abstract

Social cognition (SC), the mental operations underlying social functioning, are impaired in schizophrenia. Their direct link to functional outcome and illness status have made them an important therapeutic target. However, no effective treatment for these deficits is currently applied as a standard of care. To address this need, we have developed SocialVille—an online, plasticity-based training program that targets SC deficits in schizophrenia. Here we report the outcomes of a double-blind, controlled, randomized, multi-site clinical trial of SocialVille. Outpatients with schizophrenia were randomized to complete 40 sessions of either SocialVille (N = 55 completers) or active control (computer games; N = 53 completers) from home. The a priori co-primary outcome measures were a social cognitive composite and a functional capacity outcome (UCSD Performance-based Skills Assessment [UPSA-2]). Secondary outcomes included a virtual functional capacity measure (VRFCAT), social functioning, quality of life, and motivation. Linear mixed models revealed a group × time interaction favoring the treatment group for the social cognitive composite (b = 2.81; P < .001) but not for the UPSA-2 measure. Analysis of secondary outcome measures showed significant group × time effects favoring the treatment group on SC and social functioning, on the virtual functional capacity measure and a motivation subscale, although these latter findings were nonsignificant with FDR correction. These results provide support for the efficacy of a remote, plasticity-based social cognitive training program in improving SC and social functioning in schizophrenia. Such treatments may serve as a cost-effective adjunct to existing psychosocial treatments.

Trial Registration: NCT02246426.

Keywords: computerized, computer-based, treatment, emotion, SocialVille, TRuSST

Introduction

Social cognition (SC) refers to mental operations underlying social information processing.1–4 Multiple studies have shown that schizophrenia is associated with significant deficits in all core domains of SC,5–8 ranging from emotion and social cue perception8,9 through theory of mind (ToM) and empathy.7,8,10 These deficits have functional and clinical significance, as they underlie most critical factors of daily living in schizophrenia.11–13 Moreover, the degree of SC impairment is a stronger predictor of everyday function than are cognitive abilities or the severity of positive symptoms,14,15 making them an important therapeutic target.

However, there are currently no treatment methods that are broadly administered for improving SC in schizophrenia. Pharmacological treatments have only limited impact on SC in schizophrenia.16–18 Similarly, new interventions for treating general cognitive deficits in schizophrenia have also shown only modest impact on social functioning.19 Targeted SC interventions may be necessary to drive changes in SC, which may ultimately impact functioning. Most of these existing interventions are administered by trained professionals in small groups in clinics and usually focus on emotion management and social skill-building. Indeed, targeted interventions such as the Social Cognition and Interaction Training (SCIT),20,21 the Social Cognitive Skills Training (SCST)22 and the Social Cognition Enhancement Training (SCET)23 have all shown some promise in improving SC in schizophrenia, but there is limited evidence to positive changes in functional outcome (see refs.24–27).

Unfortunately, despite their relative success, these interventions are far from becoming standard of care, potentially due to their limited scalability, stemming from the requirement for highly trained personnel and frequent clinic visits. Computerized interventions, which are rather extensively used in cognitive training, may help overcome these difficulties, as they can be delivered remotely. Still, computer-aided interventions have been rather sparsely used to target SC, potentially because there is limited face validity of computer training for social behavior, which naturally involves other people.28 The few computerized SC programs that exist are often limited in scope—targeting a subset of SC domains—and have undergone only initial testing to date.24,29,30 Furthermore, most interventions still require some mediation by professionals and have not been applied completely remotely.

Here, we test the efficacy of SocialVille, an online intervention targeting SC abilities using individualized SC exercises. SocialVille, therefore, takes a different approach than most prior social treatment programs by using principles derived from cognitive training and neuroplasticity.31,32 As such, it is designed to engage the neural systems that support SC, which, in turn, leads to improved social skills.28 For this approach, the use of a computer is a substantial advantage: the program consists of multiple training exercises targeting multiple SC abilities, ranging from lower-level affect and social cue perception, to higher level mentalizing, self-referential style, and empathy.33 In order to promote neuroplasticity, training is intensive, adaptive, and reinforcing31; In SocialVille, each exercise requires the user to make hundreds of discriminations of socially relevant information that gradually involve more complex, multi-modal, and ecologically valid stimuli. To date, SocialVille has shown feasibility and initial efficacy in improving SC abilities in a pilot study in schizophrenia,33 and when applied in combination with standard cognitive training in at-risk34 and chronic35 schizophrenia samples. In addition, SocialVille training was shown to improve empathy in young healthy adults.36

We conducted a double-blind, multi-site randomized controlled trial (RCT), comparing the efficacy of SocialVille training to an active control (computer games), both applied remotely from home, using internet-connected laptops. Such remote application should facilitate delivery of the intervention and increase scalability. We hypothesized that experimental group participants would show larger SC and functional capacity gains, relative to the active control group.

Methods

The full protocol of this study has been published elsewhere37; most relevant details of study procedures are described below.

Participants

Clinically stable adults with schizophrenia were recruited from multiple sites: San Francisco VA Medical Center (SFVAMC), University of Minnesota (UMN), University of California, Los Angeles (UCLA), Los Angeles VA Hospital (LAVA) and Rush University. Study participants met DSM-V criteria for a diagnosis of schizophrenia (assessed using the Structured Clinical Interview for DSM-IV; SCID-P38), were between 18 and 65 years of age, had an estimated IQ ≥ 70 based on the Wechsler Test of Adult Reading (WTAR39), were clinically stable for 8 weeks prior to consent, had no more than a moderate severity rating on hallucinations and unusual thought content (a score of ≤4 on the Positive and Negative Syndrome Scale; PANSS40), and no active suicidal ideation with specific plan and intent (measured by the Columbia-Suicide Severity Rating Scale; C-SSRS41). Finally, participants had been maintained on a stable treatment of no more than 2 antipsychotics and/or other concomitant psychotropic treatment for at least 6 weeks prior to consent.

Study Design

Institutional review board approval was obtained at the coordinating center (WIRB Pro Number 20141695; ClinicalTrials.gov Identifier: NCT02246426) and at each trial site. All participants signed an informed consent form prior to participation in the study.

Eligible participants completed baseline assessments in the lab/clinic and were then randomly assigned to either experimental (SocialVille) or active control (casual games) training conditions (see below). Random allocation was performed using stratification by gender, education (<13 y, >13 y), and age (18–40 y, 41–65 y) to each group at each site with an allocation ratio of 1:1. To minimize the imbalance between the number of participants in each group over these factors, we employed a minimization method42 of adaptive stratification.

Participants were loaned a laptop and were asked to complete training from home for 3–5 times/wk, for a total of 40 training sessions (42 min each) over 8–12 weeks. After 16 weeks, they were asked to complete the post-training assessments, regardless of the amount of training completed. Training coaches interacted with participants weekly by phone in order to discuss progress and provided coaching if a participant indicated difficulty in completing training. Assessment battery was repeated in the lab mid-way through training and at the completion of the entire training program (post-training assessment).

Participants were compensated for their participation in the study, receiving $200 for all screening and assessment visits, $5 for each training session completed, and a $10 bonus for every 10 training sessions completed (maximum bonus: $40).

Interventions

Both interventions were deployed on an online, browser-playable platform by Posit Science. Full list of exercises is given in supplementary material 1. Participants completed 7 unique exercises/games on every training session, for about 6 minutes each, for a total of 42 minutes per session.

The SocialVille Training Program

Socialville is a computerized SC training program developed by Posit Science (see refs.33,37). It is composed of 27 unique exercises, which collectively target visual and vocal affect perception, social cue perception, ToM, self-referential style, and empathy. Training exercises are built using similar mechanisms to those of general cognitive training, but employ socially relevant stimuli, designed to improve processing speed and accuracy in the brain systems dedicated to the processing of social information.32

Active Control Training Program

We used 13 conventional, progressive, and commercially available computer games, which were shown to provide face-valid cognitive stimulation and were rated E (for everyone) by the Entertainment Software Rating Board (ESRB). Games have been embedded in the Posit Science training portal to help maintain blinding and control for potential placebo effects, as all participants underwent the same procedure. In addition, this type of control helped match expectation-based influences on performance as well as the experimental program in overall program use intensity, staff interaction, reward, and overall engagement. Importantly, these games did not have any social content or individualized progression.

Outcome Measures

Primary Outcome Measures

We used a co-primary outcome measure, composed of an SC composite, and a functional capacity measure. The a priori co-primary SC outcome measure was a composite score of 6 SC assessments, collectively assessing facial emotion recognition (The Penn Emotional Recognition Test, ER4043), prosody identification (The Prosody Identification Test, PROID44), immediate and delayed memory for faces (the Penn Faces Memory Test, PFMT),43 the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT45) managing emotions subscale and the Empathic Accuracy (EA)Task.46 The a priori functional capacity outcome measure was the UCSD Performance-based Skills Assessment (UPSA-2),47 which assesses skills in 5 areas: household chores, communication, finance, transportation, and planning recreational activities.

Secondary Outcome Measures

In addition to the primary outcomes, we assessed clinical status and symptom severity using the Positive and Negative Syndrome Scale (PANSS),40 and functioning using the Virtual Reality Functional Capacity Assessment Tool (VRFCAT48), a virtual reality (VR) measure mimicking a real-life scenario of a shopping trip. Functioning was further assessed using the Global Functioning Scale: Social and Role (GFS49,50), Social Functioning Scale (SFS51), an abbreviated version of the Quality of Life Scale (QLS52), and the Specific Levels of Functioning Scale (SLOF53).

SC was assessed using additional measures of facial affect perception (the Morphed Faces Task),54 social perception (The Awareness of Social Inference Test, Part 3 [TASIT])55; ToM (the Faux Pas Recognition Test56,57), memory for the source of items (The Source Memory Test58), and attributional style (The Ambiguous Intentions Hostility Questionnaire, AIHQ59).

Finally, we assessed motivation, which has been found to be a critical mediator between SC and function in schizophrenia,60 using both the Temporal Experience of Pleasure Scale (TEPS61) and the Behavioral Inhibition/Behavioral Activation Scale (BIS/BAS62), which assesses sensitivity to anticipated punishment or reward.

Data Analysis

Power calculations are reported elsewhere.37

Primary Analysis

Groups were compared on all baseline measures. Differences between groups were tested via Mann-Whitney U tests (for continuous variables) or Pearson chi-square tests (for categorical variables). Raw scores of the 6 primary SC outcomes were first converted into Z-scores, and then summed and normalized (mean of 100, SD of 15).

Since there were group differences on the primary outcomes at baseline, we implemented a propensity score framework based on demographic and primary outcomes to compare trajectories of change over time.63,64 This process effectively creates a matched sample.65

To test whether groups differed in change over time, data were analyzed according to the intent-to-treat (ITT) principle, in which all subjects randomized into either the treatment or control group were included in the analysis. This was accomplished via a linear mixed-effects model (LMM), where missing data was handled via full information maximum likelihood (FIML). FIML is a “gold standard” approach to handling missing data, assuming that data is missing at random (MAR). To examine whether this assumption may have been met, prior to running the primary analysis, we examined patterns of missing data via pattern-mixture models.66 The LMM included fixed effects of time, group, and a group × time interaction, with random intercepts of subject and site, as well as a random slope of time. Statistical significance was assessed at a 2-sided P-value of P < .05. We applied a Bonferroni adjustment to correct for multiple comparisons. Thus, the threshold for statistical significance was set at .05/2 = .025.

Results

Participants and Baseline Characteristics

A CONSORT diagram of enrollment and allocation is shown in figure 1. Recruitment began in April 2015; the final participant completed post-training assessment in February 2018.

Fig. 1.

Fig. 1.

CONSORT diagram for the randomized controlled trial. CTRL = control group; SCV = SocialVille group.

Demographic characteristics are presented in table 1 and medication regimens are shown in supplementary table 1. Groups were well-matched on demographic characteristics and estimates of premorbid intellectual abilities. Given the baseline differences between the groups on primary outcomes, we implemented propensity score weighting (supplementary material 2), which reduced the median standardized difference between groups from d = .11 at baseline to d = .03 after weighting.

Table 1.

Demographic and Clinical Characteristics for Socialville (SCV; n = 76) and Control (CTRL; n = 71) Groups

SCV Group (n = 76) CTRL Group (n = 71) t or X2 P Value
Age (y) 42.5 (13.9) 43.27 (11.5) .05 .82
Male (n, %) 53 (69.7) 49 (69) .01 .92
Education (y) 13.39 (2.3) 13.08 (1.64) .12 .73
Age at diagnosis (y) 23.5 (9.5) 23.2 (9.2) .05 .82
Race, n (%)
 Caucasian 36 (47.4) 38 (53.5) .56 .46
 African-American 30 (39.5) 29 (40.9) .03 .86
 Other 10 (13.2) 4 (5.6) 2.41 .12
IQ Estimate (WTAR) 98 (11.46) 97.5 (11.46) .10 .75
PANSS
 PANSS Positive 15.14 (4.84) 14.82 (5.34) .24 .62
 PANSS Negative 16.96 (6.28) 16.23 (5.96) .38 .54
 PANSS General 30.05 (7.78) 30.23 (7.89) .03 .86
 PANSS Total 62.16 (15.02) 61.27 (15.55) .218 .64
Baseline SC and functional capacity
 Baseline SC composite 97.68 (14.98) 102.67 (14.66) 4.14 .042*
 Baseline UPSA-2 34.18 (6.33) 36.45 (6.42) 4.89 .027*

Note: Means and SEMs are given. PANSS, Positive and Negative Syndrome Scale; SC, social cognition; UPSA-2, UCSD Performance-based Skills Assessment; WTAR, Wechsler Test of Adult Reading.

*Values are significant (P < .05).

Compliance and Feasibility of the SocialVille Training Program

Attrition Rate From the SocialVille Group

As can be seen in figure 1, 21 of the 76 participants (27.6%) randomized to the SocialVille group and 18/71 (23.6%) randomized to the control group dropped out from the study. There was no difference between study sites in terms of dropout rates (x32 =4.67, P = .198). Those who did not provide subsequent data following the baseline assessment did not differ in terms of age, gender, education, race/ethnicity, or on the UPSA-2 (all P ≥ .16). However, they performed moderately worse on the total SC composite (x2 = 5.02, P = .025, Cohen’s d approximation = .47). This suggests that our results reflect findings based on a slightly less severe population. There was minimal missing data among subjects that returned for follow-up assessments (10%), and sensitivity analyses via pattern-mixture models did not suggest that patterns of missing data influenced results among these subjects.

Most participants who dropped out from the study were unresponsive to calls made to them (n = 11) or just stopped training without providing a reason (n = 8). Additional reasons for dropping out were: difficulty completing training (n = 5), technological difficulties (eg, Wi-Fi problems; n = 3), medical complications unrelated to SZ diagnosis (n = 3), moving or change of job (n = 2), stopped attending the clinic (n = 2), did not train enough before completion of study (n = 2) or worsening of symptoms (n = 1). Finally, 3 participants were withdrawn by the investigators.

Compliance With Training

Participants were asked to complete 40 training sessions over 8–12 weeks. On average, all participants randomized to the SocialVille group completed 27.2 ± 13.3 daily sessions, and participants in the control group completed 25.5 ± 12.5 daily sessions (t = −0.94; P = .34). Study completers completed 33.8 ± 8.1 and 29.6 ± 9.5 daily sessions of SocialVille and control training, respectively.

Primary Outcomes

Growth Models

Likelihood ratio tests and Bayesian Information Criterion indicated that linear trajectories of time fit the data best for both co-primary outcomes. In terms of unconditional growth models, there was a significant increase in total SC composite scores over time (b = 3.33, |z| = 6.10, P < .001). Effect size estimates (the estimated effect size for within-subject change over time was defined as Δ/δΔ where Δ is the rate of change and δΔ is the standard deviation of the rate of change. The group × time effect size was defined as (ΔSCV/δΔSCV)(ΔC/δΔC), where SCV refers to the experimental group and C refers to the control group) for within-subject change was .51 for the total composite score and .54 for UPSA-2 scores, which are in the “moderate” range.

Results of Intent-to-Treat Analyses

Findings from the ITT models are shown in table 2. There was a significant group × time interaction on the total SC composite score, such that participants in the experimental group exhibited greater change over time than control group, in the moderate-large range (estimated Cohen’s d = .65; figure 2, left). However, there was no group × time interaction on UPSA-2 scores (figure 2, right). There were no significant group × time × baseline performance interactions (all P ≥ .08). Actual outcome values are provided in supplementary table 2.

Table 2.

Intent-to-Treat Analysis Using Linear Mixed-Effects Models

SC Composite UPSA-2
Variable B (SE) |z| P 95% CI B (SE) |z| P 95% CI
Time 1.78 (.48) 3.70 < .001 .84, 2.73 1.33 (0.49) 2.69 .007 0.36, 2.30
Group −4.08 (3.98) 1.03 .305 −11.89, 3.72 −1.78 (1.44) 1.23 .217 −4.60, 1.05
Time × Group 2.81 (.49) 5.78 < .001 1.86, 3.76 −0.14 (0.56) 0.24 .808 −1.24, 0.97

Note: SC, social cognition; UPSA-2, UCSD Performance-based Skills Assessment.

Fig. 2.

Fig. 2.

Group differences in SC composite (left) and UPSA-2 (right) scores over time, in SocialVille (solid lines) and control (dashed lines) groups. Error bars denote standard error of mean.

We conducted additional analyses on the primary outcomes (supplementary material 2). We found that change on the UPSA was moderated by baseline UPSA-2 scores, and that change on the SC composite was correlated with change in SocialVille training ToM exercises. However, there was no association between compliance rates and changes on primary outcomes. In addition, a Principal Component Analysis (PCA) conducted on the primary SC outcome showed that the SC composite had 2 components and that the component that included all outcomes except for memory for faces (PROID, ER40, MSCEIT, and Empathic Accuracy) was the main contributor to the group × time interaction.

Secondary Outcomes

Secondary outcomes were examined using the same analytic strategy outlined above. Results were corrected for multiple comparisons and P-values were adjusted using the Benjamini-Hochberg False Discovery Rate (FDR). These results are summarized in table 3.

Table 3.

Analysis of Secondary Outcome Measures

Domain (Test) Variable b |z| P-Value
Social Functioning & SFS - Engagement/Withdrawal 1.17 6.68 .000*
SFS - Interpersonal Communication −2.42 5.48 .000*
SFS - Independence Competence −1.17 2.34 .019
SFS - Prosocial 1.04 0.81 .418
SFS - Recreation 0.70 0.80 .424
SFS - Employment 0.73 0.71 .478
SFS - Independence Performance 1.00 1.92 .055
Morphed Faces Task & Total Accuracy 0.02 2.90 .004*
Global Functioning & GFS - Social 0.25 2.62 .009
GFS - Role 0.07 0.33 .741
Functional Capacity & VRFCAT Total −57.53 2.24 .025
Quality of Life & QLS - Motivation 0.19 2.23 .026
QLS - Role 0.05 1.19 .234
QLS - Anhedonia −0.13 1.10 .271
QLS - Purpose 0.11 0.64 .522
QLS - Interpersonal 0.09 0.34 .734
QLS - Curiosity −0.08 0.32 .749
QLS - Social Interaction −0.02 0.15 .881
QLS - Empathy −0.01 0.11 .912
QLS - Commonplace 0.00 0.04 .968
TASIT & TASIT - TOTDO 0.15 2.58 .010
TASIT - TOTFEEL 0.21 1.03 .303
TASIT - TOTTHINK −0.20 0.89 .373
TASIT - TOTSAY −0.03 0.13 .897
Attribution Bias (AIHQ) AIHQ - Item .25 .71 .477
AIHQ - HB .27 .48 .634
AIHQ - AB .06 .29 .773
ToM (Faux Pas) Faux Pas Total 0.00 0.05 .960
Source Memory Test Average Hit Rate 0.00 0.10 .897
Motivation TEPS - Total −0.70 0.70 .459
BIS/BAS - BAS Total 0.25 1.20 .250
BIS/BAS – BIS −0.15 0.52 .603

Note: AIHQ, The Ambiguous Intentions Hostility Questionnaire; BIS/BAS, Behavioral Inhibition/Behavioral Activation Scale; SFS, Social Functioning Scale; GFS, Global Functioning Scale; VRFCAT, Virtual Reality Functional Capacity Assessment Tool; QLS, Quality of Life Scale; ToM, theory of mind; TASIT, The Awareness of Social Inference Test; TEPS, Temporal Experience of Pleasure Scale. P-values represent the group × time interactions. Those marked with * are those P values < .05 with FDR. Those marked in bold are significant (P < .05) but did not survive FDR. Those favoring the SocialVille group are marked with a preceding &.

Significant group × time effects favoring the experimental group were found for the SFS - withdrawal subscale (b = 1.17, |z| = 6.68, P < .001), and for a facial affect recognition task (Morphed Faces Task; b = 0.02, |z| = 2.9, P = .004). However, the SFS interpersonal communication subscale showed an effect favoring the control group (b = −2.42, |z| = 5.48, P < .001).

In addition, significant effects favoring the experimental group, but that did not survive the FDR correction, were found on a functional capacity outcome (VRFCAT; b = −57.53, |z| = 2.24, P = .025), the GFS - Social (b = 0.25, |z| = 2.62, P = .009), the QLS - motivation subscale (b = 0.19, |z| = 2.23, P = .026) and the TASIT - TO DO subscale (b = 0.15, |z| = 2.58, P = .01). Significant effects favoring the control group which did not survive FDR correction were found only for the SFS independent competence subscale (b = −1.17, |z| = 2.34, P = .019).

Discussion

We conducted an RCT to test the efficacy of an online SC training program in outpatients with schizophrenia. Our results show that, compared to an active control condition, SocialVille group participants showed greater improvement on independent behavioral composite measures of SC. The improvement on the functional capacity outcome (UPSA-2) was similar in both groups. However, only the experimental group showed significant improvements on the secondary outcomes of SFS - withdrawal and on a facial affect recognition measure. In addition, SocialVille group participants improved on a VR functional capacity measure (VRFCAT), in social functioning (GFS - social), on the SC measure of TASIT (TO DO subscale) and on a clinician-rated motivation subscale of the QLS. No change was seen on symptoms or self-report measures of motivation.

These results add to those of previous studies of SocialVille, which showed SC benefits following training in a small, uncontrolled study33,34 and in a controlled study in young healthy adults.36 In addition, applying SocialVille in a combination with “cold” cognitive training34,35 or with other types of therapy67 yielded improved SC function compared with control intervention. The current study extends these results and allows for evaluation of this remotely administered, targeted SC intervention.

To our knowledge, this is the only RCT to report the results of an online SC training compared to an active control. Most reported SC interventions in schizophrenia were applied in group settings in the clinic, managed by clinicians.68 Computerized interventions to date are mainly computer-assisted (eg,69–71) and were applied in the clinic, supervised by trained personnel as part of group sessions within a broader context (eg,30,69,72). Only one other study used a remote online SC intervention of emotion perception and ToM (eMotion training).73 However, the study included treatment-as-usual rather than an active control, which makes it difficult to account for placebo effects.74

The effects found here are comparable to those reported in a recent review of manualized, group-based SC interventions,75 reporting medium-to-large effect sizes for affect identification, mentalizing, and social perception. Our results are similar to those reported for the eMotion training,76 which showed improvements in emotion recognition and some aspects of ToM. Collectively, these results show that an individualized online program can drive benefits in standardized SC outcomes.

Our results regarding the benefit of training on functional capacity were mixed. On the one hand, there were no group differences in improvements in the co-primary outcome UPSA-2. This result is in line with the null effects found for UPSA-2 in SC intervention studies (refs.22,77,78; see also refs.26,27). This could be due to the nature of the assessment itself, which may not be sensitive to detected changes in performance, or due to its tasks being outdated relative to the large technological advances characterizing the modern world. Additional concurrent treatments may be required in order to drive generalizable real-world functioning benefits.79

On the other hand, only the experimental group improved on a novel measure of functional capacity, VRFCAT, although this effect did not survive FDR correction. The VRFCAT itself has shown good validity and correlation with standard tests of functional capacity and with occupational status.80–82 These results are in line with studies showing a strong link between SC and functional outcome in schizophrenia.14

Analysis of secondary outcomes reveals a more complex picture. Significant effects were found for facial affect perception (Morphed Faces task) and for everyday social functioning (SFS - withdrawal subscale), but not for a written ToM assessment (Faux Pas) or an attributional bias questionnaire (AIHQ), both of which were reported to have small-moderate effect sizes in previous reviews.75 In addition, some outcomes were statistically significant, but lost significance following FDR correction due to the large number of outcomes, and hence should be interpreted with caution. These include social awareness (the “to do” subscale of TASIT), the GFS-social, and the motivation subscale of the QLS. This suggests that targeting SC deficits may improve some form of motivation behavior,60,83 and that this improvement may translate to improved functioning.35

This is the largest controlled SC training trial testing a fully remote intervention to date and the results support the efficacy of SocialVille as a viable means to improve SC in patients with established schizophrenia. Specifically, it addresses some of the limitations of previous SC intervention studies, including small sample size, lack of appropriate controls, limited blinding of assessors, and limitations of outcome measures.27 Still, our study has several limitations that should be controlled for in future studies. These include the lack of follow-up, which precludes us from inferring about durability of the effects. Furthermore, participants received monetary incentives for participation, which could serve as external motivation, making it difficult to infer about the usefulness of the intervention in the real-world. In addition, training was performed using loaned laptop devices, which may not be available for all patients in real life. The relatively high attrition rate (±30%) is similar to that seen in other SC training studies (eg, refs.23,69,84). However, it calls for improved methodologies to keep participants engaged and potential modification of training requirements. In addition, the fact that those who dropped out had worse SC performance at baseline may indicate that these people may have found training more challenging or difficult to complete. Training deployed online or on mobile devices may help increase usability, as training can be performed also outside the home setting.85,86

Our study importantly shows that an individualized online intervention may be efficacious in schizophrenia and may serve as an adjunct to psychosocial and/or pharmacological treatments. Surprisingly, despite the increase in use of computers and mobile devices by large segments of the population, there have been only a few attempts to develop a fully computerized SC intervention. The current study may, therefore, serve as another step in the direction of integrating computerized interventions as part of the therapeutic regime of patients with chronic schizophrenia.

Supplementary Material

sbaa085_suppl_Supplementary_Material_1
sbaa085_suppl_Supplementary_Material_2
sbaa085_suppl_Supplementary_Table_1
sbaa085_suppl_Supplementary_Table_2

Acknowledgments

We would like to thank Dr. Maurice Fried, Abby Rowlands, Lisa Howard, Jessica Dow, Connie Ludwig, Martin Hanson, Shasteana Rancher, Ariel Currie, Riley Capizzi, Gabrielle Steinhoff, Chetana Guthikonda, Emily Johnson, Sarah Pridgen, Abhishek Saxena and Briana Galindo for their help with data collection. Authors H.L., M.M.M., and S-J.K. are paid employees and A.R. serves as a consultant and holds equity in Posit Science, the company which holds rights for the computerized SocialVille training, which is described in the manuscript.

Funding

Research reported in this publication was supported by the National Institute of Mental Health (NIMH) Award R44MH091793 and by the National Center for Advancing Translational Sciences of the National Institutes of Health (Award Number UL1-TR002494). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

References

  • 1. Augoustinos M, Walker I, Donaghue N.. Social Cognition: An Integrated Introduction, 2nd edition. London, UK: SAGE Publications Ltd; 2014. [Google Scholar]
  • 2. Fiske ST, Taylor SE.. Social Cognition: From Brains to Culture. Boston, MA: McGraw-Hill Higher Education; 2008. [Google Scholar]
  • 3. Brothers L. The social brain: a project for integrating primate behavior and neurophysiology in a new domain. Concepts Neurosci 1990;1:27–51. [Google Scholar]
  • 4. Adolphs R. Cognitive neuroscience of human social behaviour. Nat Rev Neurosci. 2003;4(3):165–178. [DOI] [PubMed] [Google Scholar]
  • 5. Green MF, Horan WP, Lee J. Social cognition in schizophrenia. Nat Rev Neurosci. 2015;16(10):620–631. [DOI] [PubMed] [Google Scholar]
  • 6. Kohler CG, Walker JB, Martin EA, Healey KM, Moberg PJ. Facial emotion perception in schizophrenia: a meta-analytic review. Schizophr Bull. 2010;36(5):1009–1019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Bora E, Yucel M, Pantelis C. Theory of mind impairment in schizophrenia: meta-analysis. Schizophr Res. 2009;109(1–3):1–9. [DOI] [PubMed] [Google Scholar]
  • 8. Savla GN, Vella L, Armstrong CC, Penn DL, Twamley EW. Deficits in domains of social cognition in schizophrenia: a meta-analysis of the empirical evidence. Schizophr Bull. 2013;39(5):979–992. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Bortolon C, Capdevielle D, Raffard S. Face recognition in schizophrenia disorder: a comprehensive review of behavioral, neuroimaging and neurophysiological studies. Neurosci Biobehav Rev. 2015;53:79–107. [DOI] [PubMed] [Google Scholar]
  • 10. Harvey PO, Zaki J, Lee J, Ochsner K, Green MF. Neural substrates of empathic accuracy in people with schizophrenia. Schizophr Bull. 2013;39(3):617–628. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Green MF, Nuechterlein KH. Should schizophrenia be treated as a neurocognitive disorder? Schizophr Bull. 1999;25(2):309–319. [DOI] [PubMed] [Google Scholar]
  • 12. Albert N, Bertelsen M, Thorup A, et al. Predictors of recovery from psychosis Analyses of clinical and social factors associated with recovery among patients with first-episode psychosis after 5 years. Schizophr Res. 2011;125(2–3):257–266. [DOI] [PubMed] [Google Scholar]
  • 13. Couture SM, Penn DL, Roberts DL. The functional significance of social cognition in schizophrenia: a review. Schizophr Bull. 2006;32(suppl 1):S44–S63. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Green MF. Impact of cognitive and social cognitive impairment on functional outcomes in patients with schizophrenia. J Clin Psychiatry. 2016;77(suppl 2):8–11. [DOI] [PubMed] [Google Scholar]
  • 15. de Jong JJ, de Gelder B, Hodiamont PP. Sensory processing, neurocognition, and social cognition in schizophrenia: towards a cohesive cognitive model. Schizophr Res. 2013;146(1–3):209–216. [DOI] [PubMed] [Google Scholar]
  • 16. Keefe RS, Bilder RM, Davis SM, et al. ; CATIE Investigators; Neurocognitive Working Group. Neurocognitive effects of antipsychotic medications in patients with chronic schizophrenia in the CATIE Trial. Arch Gen Psychiatry. 2007;64(6):633–647. [DOI] [PubMed] [Google Scholar]
  • 17. Mueser KT, Doonan R, Penn DL, et al. Emotion recognition and social competence in chronic schizophrenia. J Abnorm Psychol. 1996;105(2):271–275. [DOI] [PubMed] [Google Scholar]
  • 18. Green MF. Stimulating the development of drug treatments to improve cognition in schizophrenia. Annu Rev Clin Psychol. 2007;3:159–180. [DOI] [PubMed] [Google Scholar]
  • 19. Sacks S, Fisher M, Garrett C, et al. Combining computerized social cognitive training with neuroplasticity-based auditory training in schizophrenia. Clin Schizophr Relat Psychoses. 2013;7(2):78–86A. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Combs DR, Elerson K, Penn DL, et al. Stability and generalization of Social Cognition and Interaction Training (SCIT) for schizophrenia: six-month follow-up results. Schizophr Res. 2009;112(1–3):196–197. [DOI] [PubMed] [Google Scholar]
  • 21. Roberts DL, Penn DL. Social cognition and interaction training (SCIT) for outpatients with schizophrenia: a preliminary study. Psychiatry Res. 2009;166(2–3):141–147. [DOI] [PubMed] [Google Scholar]
  • 22. Horan WP, Dolinsky M, Lee J, et al. Social cognitive skills training for psychosis with community-based training exercises: a randomized controlled trial. Schizophr Bull. 2018;44(6):1254–1266. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Choi KH, Kwon JH. Social cognition enhancement training for schizophrenia: a preliminary randomized controlled trial. Community Ment Health J. 2006;42(2):177–187. [DOI] [PubMed] [Google Scholar]
  • 24. Kurtz MM, Richardson CL. Social cognitive training for schizophrenia: a meta-analytic investigation of controlled research. Schizophr Bull. 2012;38(5):1092–1104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Tan BL, Lee SA, Lee J. Social cognitive interventions for people with schizophrenia: a systematic review. Asian J Psychiatr. 2018;35:115–131. [DOI] [PubMed] [Google Scholar]
  • 26. Grant N, Lawrence M, Preti A, Wykes T, Cella M. Social cognition interventions for people with schizophrenia: a systematic review focussing on methodological quality and intervention modality. Clin Psychol Rev. 2017;56:55–64. [DOI] [PubMed] [Google Scholar]
  • 27. Horan WP, Green MF. Treatment of social cognition in schizophrenia: current status and future directions. Schizophr Res. 2019;203:3–11. [DOI] [PubMed] [Google Scholar]
  • 28. Dodell-Feder D, Tully LM, Hooker CI. Social impairment in schizophrenia: new approaches for treating a persistent problem. Curr Opin Psychiatry. 2015;28(3):236–242. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29. Frommann N, Streit M, Wölwer W. Remediation of facial affect recognition impairments in patients with schizophrenia: a new training program. Psychiatry Res. 2003;117(3):281–284. [DOI] [PubMed] [Google Scholar]
  • 30. Wölwer W, Frommann N, Halfmann S, Piaszek A, Streit M, Gaebel W. Remediation of impairments in facial affect recognition in schizophrenia: efficacy and specificity of a new training program. Schizophr Res. 2005;80(2–3):295–303. [DOI] [PubMed] [Google Scholar]
  • 31. Fisher M, Mellon SH, Wolkowitz O, Vinogradov S. Neuroscience-informed auditory training in schizophrenia: a final report of the effects on cognition and serum brain-derived neurotrophic factor. Schizophr Res Cogn. 2016;3:1–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32. Nahum M, Lee H, Merzenich MM. Principles of neuroplasticity-based rehabilitation. Prog Brain Res. 2013;207:141–171. [DOI] [PubMed] [Google Scholar]
  • 33. Nahum M, Fisher M, Loewy R, et al. A novel, online social cognitive training program for young adults with schizophrenia: a pilot study. Schizophr Res Cogn. 2014;1(1):e11–e19. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34. Hooker CI, Carol EE, Eisenstein TJ, et al. A pilot study of cognitive training in clinical high risk for psychosis: initial evidence of cognitive benefit. Schizophr Res. 2014;157(1–3):314–316. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Fisher M, Nahum M, Howard E, et al. Supplementing intensive targeted computerized cognitive training with social cognitive exercises for people with schizophrenia: an interim report. Psychiatr Rehabil J. 2017;40(1):21–32. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Haut KM, Dodell-Feder D, Guty E, Nahum M, Hooker CI. Change in objective measure of empathic accuracy following social cognitive training. Front Psychiatry. 2019;10:894. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Rose A, Vinogradov S, Fisher M, et al. Randomized controlled trial of computer-based treatment of social cognition in schizophrenia: the TRuSST trial protocol. BMC Psychiatry. 2015;15:142. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. First MB, Spitzer RL, Williams JBW. Structured Clinical Interview for DSM-IVTR Axis I Disorders, Research Version, Patient Edition. (SCID-I/P). New York: Biometrics Research, New York Psychiatric Institute; 2002. [Google Scholar]
  • 39. Holdnack J. WTAR: Wechsler Test of Adult Reading manual. San Antonio TX: Psychol. Corp.; 2001. [Google Scholar]
  • 40. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull. 1987;13(2):261–276. [DOI] [PubMed] [Google Scholar]
  • 41. Posner K, Brown GK, Stanley B, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011;168(12):1266–1277. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. Endo A, Nagatani F, Hamada C, Yoshimura I. Minimization method for balancing continuous prognostic variables between treatment and control groups using Kullback-Leibler divergence. Contemp Clin Trials. 2006;27(5):420–431. [DOI] [PubMed] [Google Scholar]
  • 43. Gur RC, Ragland JD, Moberg PJ, et al. Computerized neurocognitive scanning: I. Methodology and validation in healthy people. Neuropsychopharmacology. 2001;25(5):766–776. [DOI] [PubMed] [Google Scholar]
  • 44.Russ J. Development of PROID, a computerized emotional prosody identification task. PennScience J. 2008;6:14–20. [Google Scholar]
  • 45. Mayer JD, Salovey P, Caruso DR, Sitarenios G. Measuring emotional intelligence with the MSCEIT V2.0. Emotion. 2003;3(1):97–105. [DOI] [PubMed] [Google Scholar]
  • 46. Lee J, Zaki J, Harvey PO, Ochsner K, Green MF. Schizophrenia patients are impaired in empathic accuracy. Psychol Med. 2011;41(11):2297–2304. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47. Patterson TL, Goldman S, McKibbin CL, Hughs T, Jeste DV. UCSD Performance-Based Skills Assessment: development of a new measure of everyday functioning for severely mentally ill adults. Schizophr Bull. 2001;27(2):235–245. [DOI] [PubMed] [Google Scholar]
  • 48. Ruse SA, Harvey PD, Davis VG, Atkins AS, Fox KH, Keefe RS. Virtual reality functional capacity assessment in schizophrenia: preliminary data regarding feasibility and correlations with cognitive and functional capacity performance. Schizophr Res Cogn. 2014;1(1):e21–e26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49. Auther AM, Smith CW, Cornblatt BA.. Global Functioning:Social Scale (GF: Social). Glen Oaks, NY: Zucker Hillside Hospital; 2006. [Google Scholar]
  • 50. Niendam TA, Bearden CE, Johnson JK, Cannon TD.. GlobalFunctioning: Role Scale (GF: Role). Los Angeles, CA: University of California, Los Angeles; 2006. [Google Scholar]
  • 51. Birchwood M, Smith J, Cochrane R, Wetton S, Copestake S. The social functioning scale. The development and validation of a new scale of social adjustment for use in family intervention programmes with schizophrenic patients. Br J Psychiatry. 1990;157:853–859. [DOI] [PubMed] [Google Scholar]
  • 52. Heinrichs DW, Hanlon TE, Carpenter WT Jr. The quality of life scale: an instrument for rating the schizophrenic deficit syndrome. Schizophr Bull. 1984;10(3):388–398. [DOI] [PubMed] [Google Scholar]
  • 53. Schneider LC, Struening EL. SLOF: a behavioral rating scale for assessing the mentally ill. Soc Work Res Abstr. 1983;19(3):9–21. [DOI] [PubMed] [Google Scholar]
  • 54. Germine LT, Hooker CI. Face emotion recognition is related to individual differences in psychosis-proneness. Psychol Med. 2011;41(5):937–947. [DOI] [PubMed] [Google Scholar]
  • 55. McDonald S, Bornhofen C, Shum D, Long E, Saunders C, Neulinger K. Reliability and validity of The Awareness of Social Inference Test (TASIT): a clinical test of social perception. Disabil Rehabil. 2006;28(24):1529–1542. [DOI] [PubMed] [Google Scholar]
  • 56. Stone VE, Baron-Cohen S, Knight RT. Frontal lobe contributions to theory of mind. J Cogn Neurosci. 1998;10(5):640–656. [DOI] [PubMed] [Google Scholar]
  • 57. Gregory C, Lough S, Stone V, et al. Theory of mind in patients with frontal variant frontotemporal dementia and Alzheimer’s disease: theoretical and practical implications. Brain. 2002;125(Pt 4):752–764. [DOI] [PubMed] [Google Scholar]
  • 58. Vinogradov S, Willis-Shore J, Poole JH, Marten E, Ober BA, Shenaut GK. Clinical and neurocognitive aspects of source monitoring errors in schizophrenia. Am J Psychiatry. 1997;154(11):1530–1537. [DOI] [PubMed] [Google Scholar]
  • 59. Combs DR, Penn DL, Wicher M, Waldheter E. The Ambiguous Intentions Hostility Questionnaire (AIHQ): a new measure for evaluating hostile social-cognitive biases in paranoia. Cogn Neuropsychiatry. 2007;12(2):128–143. [DOI] [PubMed] [Google Scholar]
  • 60. Gard DE, Fisher M, Garrett C, Genevsky A, Vinogradov S. Motivation and its relationship to neurocognition, social cognition, and functional outcome in schizophrenia. Schizophr Res. 2009;115(1):74–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61. Gard DE, Gard MG, Kring AM, John OP. Anticipatory and consummatory components of the experience of pleasure: a scale development study. Journal of Research in Personality 2006;40(6):1086–1102. [Google Scholar]
  • 62. Carver CS, White TL. Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: the BIS/BAS scales. J Pers Soc Psychol. 1994;67:319–333. [Google Scholar]
  • 63. Rubin DB. Using multivariate matched sampling and regression adjustment to control bias in observational studies. J Am Stat Assoc. 1979;74:318–328. [Google Scholar]
  • 64. Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. 2011;46(3):399–424. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65. Hirano K, Imbens GW. Estimation of causal effects using propensity score weighting: an application to data on right heart catheterization. Health Services and Outcomes Research Methodology 2001;2:259–278. [Google Scholar]
  • 66. Hedeker D, Gibbons R. Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychol Methods. 1997;2:64–78. [Google Scholar]
  • 67. Biagianti B, Schlosser D, Nahum M, Woolley J, Vinogradov S. Creating Live Interactions to Mitigate Barriers (CLIMB): a mobile intervention to improve social functioning in people with chronic psychotic disorders. JMIR Ment Health. 2016;3(4):e52. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68. Marsh P, Langdon R, McGuire J, Harris A, Polito V, Coltheart M. An open clinical trial assessing a novel training program for social cognitive impairment in schizophrenia. Australas Psychiatry. 2013;21(2):122–126. [DOI] [PubMed] [Google Scholar]
  • 69. Wölwer W, Frommann N. Social-cognitive remediation in schizophrenia: generalization of effects of the Training of Affect Recognition (TAR). Schizophr Bull. 2011;37(suppl 2):S63–S70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70. Palumbo D, Mucci A, Piegari G, D’Alise V, Mazza A, Galderisi S. SoCIAL - training cognition in schizophrenia: a pilot study. Neuropsychiatr Dis Treat. 2017;13:1947–1956. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71. Bechi M, Spangaro M, Bosia M, et al. Theory of Mind intervention for outpatients with schizophrenia. Neuropsychol Rehabil. 2013;23(3):383–400. [DOI] [PubMed] [Google Scholar]
  • 72. Gaudelus B, Virgile J, Geliot S, Franck N; GAÏA/RECOS Study Team Improving facial emotion recognition in schizophrenia: a controlled study comparing specific and attentional focused cognitive remediation. Front Psychiatry. 2016;7:105. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73. Maroño Souto Y, Vázquez Campo M, Díaz Llenderrozas F, Rodríguez Álvarez M, Mateos R, García Caballero A. Randomized clinical trial with e-MotionalTraining® 1.0 for social cognition rehabilitation in schizophrenia. Front Psychiatry. 2018;9:40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74. Green CS, Bavelier D, Kramer AF, et al. Improving methodological standards in behavioral interventions for cognitive enhancement. J Cognit Enhancement. 2019;3:2–29. [Google Scholar]
  • 75. Kurtz MM, Gagen E, Rocha NB, Machado S, Penn DL. Comprehensive treatments for social cognitive deficits in schizophrenia: a critical review and effect-size analysis of controlled studies. Clin Psychol Rev. 2016;43:80–89. [DOI] [PubMed] [Google Scholar]
  • 76. Vázquez-Campo M, Maroño Y, Lahera G, Mateos R, García-Caballero A. e-Motional Training®: pilot study on a novel online training program on social cognition for patients with schizophrenia. Schizophr Res Cogn. 2016;4:10–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77. Horan WP, Kern RS, Tripp C, et al. Efficacy and specificity of social cognitive skills training for outpatients with psychotic disorders. J Psychiatr Res. 2011;45(8):1113–1122. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 78. Roberts DL, Combs DR, Willoughby M, et al. A randomized, controlled trial of Social Cognition and Interaction Training (SCIT) for outpatients with schizophrenia spectrum disorders. Br J Clin Psychol. 2014;53(3):281–298. [DOI] [PubMed] [Google Scholar]
  • 79. Bowie CR, McGurk SR, Mausbach B, Patterson TL, Harvey PD. Combined cognitive remediation and functional skills training for schizophrenia: effects on cognition, functional competence, and real-world behavior. Am J Psychiatry. 2012;169(7):710–718. [DOI] [PubMed] [Google Scholar]
  • 80. Ventura J, Welikson T, Ered A, et al. Virtual reality assessment of functional capacity in the early course of schizophrenia: associations with cognitive performance and daily functioning. Early Interv Psychiatry. 2020;14(1):106–114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 81. Harvey PD, Khan A, Atkins A, Keefe RS. Virtual reality assessment of functional capacity in people with Schizophrenia: associations with reduced emotional experience and prediction of functional outcomes. Psychiatry Res. 2019;277:58–63. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 82. Keefe RSE, Davis VG, Atkins AS, et al. Validation of a Computerized test of Functional Capacity. Schizophr Res. 2016;175(1–3):90–96. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 83. Bhagyavathi HD, Mehta UM, Thirthalli J, et al. Cascading and combined effects of cognitive deficits and residual symptoms on functional outcome in schizophrenia - A path-analytical approach. Psychiatry Res. 2015;229(1–2):264–271. [DOI] [PubMed] [Google Scholar]
  • 84. Ahissar M, Hochstein S. Attentional control of early perceptual learning. Proc Natl Acad Sci U S A. 1993;90(12):5718–5722. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 85. Bonet L, Izquierdo C, Escartí MJ, et al. Use of mobile technologies in patients with psychosis: a systematic review. Rev Psiquiatr Salud Ment. 2017;10(3):168–178. [DOI] [PubMed] [Google Scholar]
  • 86. Gay K, Torous J, Joseph A, Pandya A, Duckworth K. Digital technology use among individuals with schizophrenia: results of an online survey. JMIR Ment Health. 2016;3(2):e15. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

sbaa085_suppl_Supplementary_Material_1
sbaa085_suppl_Supplementary_Material_2
sbaa085_suppl_Supplementary_Table_1
sbaa085_suppl_Supplementary_Table_2

Articles from Schizophrenia Bulletin are provided here courtesy of Oxford University Press

RESOURCES