Neurobehavioral Indices of Gaze Perception are Associated with Social Cognition across Schizophrenia Patients and Healthy Controls

Abstract

Background:

Gaze perception is a basic building block of social cognition, which is impaired in schizophrenia and contributes to functional outcomes. Few studies, however, have investigated neural underpinnings of gaze perception and their relation to social cognition. We address this gap.

Method:

We recruited 77 schizophrenia patients and 71 healthy controls, who completed various social-cognition tasks. During fMRI, participants (62 schizophrenia, 54 controls) completed a gaze-perception task, where they judged whether faces with varying gaze angles were self-directed or averted; as a control condition, participants identified stimulus gender. Activation estimates were extracted based on 1) task vs. baseline, 2) gaze-perception vs. gender-identification, 3) parametric modulation by perception of stimuli as self-directed vs. averted, and 4) parametric modulation by stimulus gaze angle. We used latent variable analysis to test associations among diagnostic group, brain activation, gaze perception, and social cognition.

Results:

Preferential activation to gaze perception was observed throughout dorsomedial prefrontal cortex, superior temporal sulcus, and insula. Activation was modulated by stimulus gaze angle and perception of stimuli as self-directed vs. averted. More precise gaze perception and higher task-related activation were associated with better social cognition. Patients with schizophrenia showed hyperactivation within left pre-/post-central gyrus, which was associated with more precise gaze perception and fewer symptoms and thus may be a compensatory mechanism.

Conclusions:

Neural and behavioral indices of gaze perception were related to social cognition, across patients and controls. This suggests gaze perception is an important perceptual building block for more complex social cognition. Results are discussed in the context of dimensional psychopathology and clinical heterogeneity.

General Scientific Summary

Our study of 77 schizophrenia patients and 71 healthy controls combined behavioral tasks, functional neuroimaging, and statistical modeling to examine how eye gaze perception—a low-level visual process—functions as a basic building block of social cognition. We found that both brain and behavioral indices of gaze perception were associated with participants’ performance across a range of social-cognition tasks (e.g., emotion recognition and emotion regulation). This suggests that gaze processing may be an important perceptual building block for more complex social cognition and related real-world functioning, which could serve as a potential target for future interventions in those with schizophrenia.

Impaired social cognition is a core feature of schizophrenia (SZ) and a critical determinant of functional outcomes (Fett et al., 2011). Limited response to pharmacological treatments (Harvey et al., 2006) has inspired considerable effort to better understand the mechanisms of social cognitive deficits in SZ, which could eventually inform the development of better biobehavioral interventions. One way to better understand social cognition (and its disruption in mental illness) is to elucidate the basic building blocks that make social cognition possible. For instance, basic visual processing deficits are common in schizophrenia and contribute to problems with social cognition and real-world functioning (Silverstein & Keane, 2011; Tso et al., 2014b). One possible bridge between low-level visual processing and social cognition is eye gaze perception, the focus of our current study.

Gaze is a ubiquitous social cue that conveys important information about one’s attention, emotion, and mental state (Emery, 2000). Gaze perception is developmentally foundational (Farroni et al., 2002) and is included as a core social process in the National Institute of Mental Health (NIMH) Research Domain Criteria (RDoC) Matrix. Understanding aberrant gaze processing in SZ—which is characterized by reduced perceptual precision, increased self-referential bias, and impaired spatial coding—may shed light on mechanisms underlying symptoms such as paranoia and delusions of reference (Abbott et al., 2018; Hooker & Park, 2005; Tso et al., 2012; Tso, Taylor, et al., 2021; Tso et al., 2014a; Röder et al., 2015). The present study aimed to elucidate neural underpinnings of gaze perception, their disruption in SZ, and their associations with social cognition. We address these questions using neuroimaging, psychophysics, and latent variable analysis.

The neural substrates of gaze perception are distributed throughout regions associated with social cognition and broader sensory processing; these include the anterior insula, anterior cingulate cortex (ACC), dorsomedial prefrontal cortex (dmPFC), temporoparietal junction (TPJ), inferior parietal lobule (IPL), superior temporal sulcus/gyrus (STS/STG), angular gyrus, posterior cingulate cortex (PCC), and fusiform gyrus (Bristow et al., 2007; Carlin et al., 2011; Cavallo et al., 2015; Grosbras et al., 2005; Hooker et al., 2003; Itier & Batty, 2009; Sato et al., 2016). Many of these regions show aberrant function in SZ (Green et al., 2015), with specific evidence of frontal, occipital, and limbic hypoactivation as well as abnormal connectivity during gaze processing (Kohler et al., 2008; Pinkham et al., 2011; Tso, Angstadt, et al., 2021). Despite the prevalence of case-control studies in schizophrenia research, there are also substantial within-group individual differences in social cognition. Given that many psychiatric symptoms and mechanisms cut across disorders and also relate to variation in normal-range traits (Insel et al., 2010; Kotov et al., 2017), it is important to understand not only how social cognition and its associated mechanisms differ between SZ and healthy controls (HC), but also to understand these associations within and across SZ and HC groups and whether any such associations differ by group.

Several studies have documented robust correlations among gaze processing, social cognition, and associated neural systems. For instance, general population studies have linked gaze perception to social cognitive ability and social functioning (Lasagna et al., 2020; McCrackin & Itier, 2021; Wastler & Lenzenweger, 2018). Gaze perception abilities and related brain activity have also been linked to broader social cognition and functional outcomes in SZ (Pinkham et al., 2011; Tso et al., 2012; Tso, Taylor, et al., 2021). Finally, research spanning HC and SZ has linked individual differences in social functioning to default network connectivity (Meda et al., 2014) and effective connectivity during gaze processing (Tso, Angstadt, et al., 2021).

Many studies only test associations among observed variables (e.g., accuracy on a single social-cognition task or brain activity within specific regions of interest), but single-task, performance-based indicators often show limited reliability (Apperly, 2012; Enkavi et al., 2019). Moreover, performance on any given task is influenced by task-specific factors that may or may not relate to the underlying construct of interest. The ability to reliably measure constructs and estimate their interrelations is improved by using latent variable analyses (Schumacker & Lomax, 2004). For example, a social cognition latent variable can be modeled as the shared variance of scores across measures of emotion recognition, mentalizing, and emotion regulation. Latent variables allow us to eliminate unsystematic error variance, facilitating more accurate estimates of variance and covariance for our constructs of interest (Enkavi et al., 2019; Blain et al., 2020; Eisenberg et al., 2019). Latent variable analysis can also incorporate fMRI data, which can be used to examine how behavioral constructs like social cognition are associated with neural factors representing shared variance of brain activity across multiple regions recruited in various processes (Cooper et al., 2019; Kim et al., 2007; Lahey et al., 2012).

The current study investigated: a) neural correlates of gaze perception in SZ and HC, b) associations between gaze perception and social cognition, c) associations between patterns of brain activity during gaze perception and social cognition, and d) whether associations among gaze perception, social cognition, and associated brain activity differ between diagnostic groups. Participants completed a comprehensive battery of social-cognition tasks, spanning emotion recognition, perceptual theory of mind, and emotion regulation. Additionally, they completed a gaze perception task during fMRI. In this task, the gaze angles of face stimuli were sampled from a continuum ranging from self-directed (looking at the participant) to averted (looking away from the participant). Building on evidence of differential brain responses to direct vs. averted gaze (Berchio et al., 2016; Boyarskaya et al., 2015; Kesner et al., 2018; Marquardt et al., 2017) we were motivated to examine how brain activity might be modulated by the degree to which each stimulus is directed toward the observer. Thus, we examined patterns of neural activation associated with gaze perception using four fMRI contrasts (i.e., activation during the task compared to baseline, during gaze perception compared to gender discrimination, and as a function of whether gaze is objectively/subjectively self-directed or averted). We then used latent variable analysis to examine how gaze perception performance and associated patterns of brain activity (i.e., exploratory latent neural factors representing shared variance in brain activation clusters across our four fMRI contrasts of interest) might be associated with diagnosis and social cognition. Follow-up analyses examined whether associations among gaze perception, social cognition, and neural factors differed between groups.

We hypothesized gaze perception would be associated with activity distributed throughout regions associated with visual processing and social cognition. Specifically, we anticipated preferential activation for gaze (vs. gender) in the insula, IPL/angular gyrus, TPJ, STS/STG, and dmPFC. We also hypothesized these regions would show attenuated activation in SZ. We hypothesized that better social cognition (i.e., shared variance in performance across relevant tasks) would be associated with gaze-perception performance (i.e., higher perceptual precision and lower self-referential bias). We also hypothesized latent neural factors—reflecting important aspects of neural response to gaze—would be associated with gaze-perception performance, social cognition, and diagnostic group. Finally, we explored whether associations among gaze perception, social cognition, and neural response to gaze differed between groups and whether these variables were associated with symptom severity within the SZ group.

Method

Participants

Seventy-seven individuals with DSM-IV-TR schizophrenia or schizoaffective disorder (SZ) and 71 healthy controls (HC) completed the study. Diagnoses were established using the Structured Clinical Interview for DSM-IV Axis-I Disorders (SCID-IV) (First et al., 1995). Demographic characteristics of SZ and HC were well matched and are summarized in Table 1. In terms of race, the sample included 88 White (59.4%), 39 Black (26.3%), 14 Asian (9.5%), one Native American (0.7%), and six participants identifying as multiracial or other (4.1%); nine participants identified as Hispanic (6.1%). Of the 148 participants with behavioral data, 116 had valid fMRI data (62 SZ, 54 HC).

Table 1.

Demographic and Clinical Characteristics

Full Sample	SZ (n = 77) Mean ± SD	HC (n = 71) Mean ± SD	t or χ2	p
Age	33.2 ± 10.0	33.2 ± 11.1	0.0	.993
Sex (male/female)	41 / 36	35 / 36	0.2	.631
Education, years	14.4 ± 2.1	16.1 ± 1.9	−5.1	< .001
Parental education, years	15.5 ± 2.5	15.6 ± 3.2	−0.1	.903
MATRICS Working Memory	43.0	53.1	−6.1	< .001
MATRICS Reasoning	45.3	51.2	−3.4	< .001
MATRICS Composite	38.2	55.3	−8.9	< .001
PANSS Positive	17.0 ± 5.7
PANSS Negative	14.6 ± 4.9
SAPS	20.8 ± 16.9
SANS	32.8 ± 17.5
CPZeq	349.5 ± 351.8
Diagnosis (Schizophrenia/Schizoaffective)	31 / 46
fMRI Sample	SZ (n = 62) Mean ± SD	HC (n = 54) Mean ± SD	t or χ2	p
Age	32.9 ± 10.0	33.0 ± 11.1	0.0	.968
Sex (male/female)	30 / 32	27 / 27	0.0	.862
Education, years	14.4 ± 2.2	16.1 ± 2.0	−4.2	< .001
MATRICS Working Memory	42.2	51.2	−4.3	< .001
MATRICS Reasoning	43.9	50.4	−3.4	< .001
MATRICS Composite	38.8	52.4	−5.5	< .001
Parental education, years	15.5 ± 2.4	15.7 ± 3.4	−0.8	.788
PANSS Positive	17.0 ± 5.6
PANSS Negative	14.6 ± 4.8
SAPS	19.8 ± 17.3
SANS	33.1 ± 17.3
CPZeq	330.4 ± 344.9
Diagnosis (Schizophrenia/Schizoaffective)	22 / 40

Note. Parental education represents an average of maternal and paternal education.

Participants were recruited through community advertisements and referrals from local clinics. Inclusion/exclusion criteria are documented in the supplement. Clinical characteristics for the SZ group, including symptoms and CPZ dose equivalents are reported in Table 1. The study was conducted in accordance with the protocol approved by the Institutional Review Board of the University of Michigan Medical School (HUM00080457, “Neural Mechanisms of Gaze Perception in Psychosis”), and written informed consent was obtained from each participant.

A subset of participants (27 SZ, 22 HC) overlaps with those from previously published studies (Tso, Angstadt, et al., 2021; Tso, Burton, et al., 2021). These studies differed substantially from the current work, with one focusing on a region-of-interest analysis of patients with bipolar disorder (who are not included in the current study) compared to healthy controls (Tso, Burton, et al., 2021) and the other focusing on effective connectivity during gaze perception (Tso, Angstadt, et al., 2021).

Assessments

Patients were assessed (Table 1) using either the Positive and Negative Syndrome Scale (PANSS) (Kay et al., 1987) or the Scale for the Assessment of Positive Symptoms and Scale for the Assessment of Negative Symptoms (SAPS/SANS) (Andreasen & Grove, 1986). Social cognition was assessed using the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) (Mayer et al., 1999), Reading the Mind in the Eyes test (RME) (Baron-Cohen et al., 2001), and Penn Emotion Recognition Task (ER-40) (Kohler et al., 2003). The MATRICS battery was used to assess various other aspects of cognitive ability (Marder & Fenton, 2004; Nuechterlein et al., 2008).

fMRI Data Acquisition and Processing

MRI scanning occurred on a 3.0 T GE MR 750 Discovery scanner. fMRI data were processed using Statistical Parametric Mapping (Ashbumer et al., 2021). See the supplement for additional scanning procedure and preprocessing details. The supplement also describes details of two subsamples (n₁ = 49, including 27 SZ and 22 HC; n₂ = 67, including 35 SZ and 32 HC) that made up the dataset, as well as data harmonization procedure for the two studies from where the subsamples came.

fMRI Paradigm: Gaze Perception Task

Stimuli were naturalistic face images with 9 different gaze angles presented in pseudorandomized order. These gaze angles encompass the psychophysical aspect of the task (Figure 1a), representing levels of eye-contact signal strength, ranging from 0.2 (averted), 0.3, …, to 1.0 (self-directed). For each face, participants indicated perceived eye contact or gender by pressing a button (Figure 1b). For the gaze condition, participants indicated whether or not each face was looking at them. For the gender condition, participants indicated whether each face was male or female. Participants in the first dataset subsample completed 108 trials (6 trials × 6 blocks × 3 runs) for each task (Eyes and Gender). Participants in the second dataset subsample completed 216 trials (6 trials × 6 blocks × 6 runs) for each task (Eyes, Gender).

Figure 1. Face Stimuli and Design of the Gaze Perception Task.

Note. a) Face stimuli with 9 gaze angles. b) Eyes and Gender trials were presented in alternating blocks with a fixation block between each task block. c) After curve-fitting, two metrics were obtained from each participant’s function: threshold (m) and width (w).

Behavioral Data Analyses

Psychophysical indices of gaze perception

For each participant, eye-contact endorsement rates across different gaze angles were modeled as a logistic function of gaze angle (i.e., objective eye-contact signal strength); two psychophysical properties (threshold and width) were derived from this curve (Figure 1c). Threshold represents how strong the eye contact signal needs to be for the individual to perceive eye contact 50% of the time. Lower thresholds indicate that weaker signals are needed to perceive gaze as directed toward oneself, thus representing stronger self-referential bias. Width represents the difference in signal strengths for which participants are endorsing 5% self-directed responses vs. 95% self-directed response. Narrower width (lower values) indicates higher perceptual precision, suggesting the participant is more sensitive to small changes in gaze angle.

Group Differences in Social Cognition

Descriptive statistics were computed for social cognition measures and performance on the gaze task; to test whether SZ and HC differed, independent samples t-tests were performed, and Cohen’s d statistics were computed. Variable distributions, sorted by group, are displayed in Figure S1.

fMRI Data Analyses

fMRI data were modeled to identify patterns of brain activity associated with aspects of gaze perception. First, the anatomically normalized time series was regressed on 2 regressors of interest (Gaze trials, Gender trials) and nuisance regressors (corresponding to motion parameters and runs), convolved with a canonical hemodynamic response function. An additional regressor was used to identify activation modulated by participants’ individual perception (as indicated by corresponding behavioral endorsement) of each stimulus as self-directed (value of 1) or averted (value of 0), with averaged values within a given block used for trials with no response. In an alternative model, we used a parametric modulation regressor to identify activation that varies as a function of stimulus gaze angle (regressor values for each trial ranged from −4 to 4 in increments of 1, with larger values corresponding to more self-directed gaze).

For our second-level models, we examined which voxel clusters were 1) commonly activated across all participants and 2) differentially activated between SZ and HC. Models were computed for all-trials vs. baseline, gaze vs. gender, modulation by participants’ perception of stimuli as self-directed, and modulation by stimulus gaze angle. Additional covariates were included corresponding to head motion (mean framewise displacement) and dataset subsample. Second-level models were computed using a cluster-defining threshold of p < 0. 001 and FWE correction of p < 0.05 (Eklund et al., 2016). After examining which clusters were significant at the whole-sample level and for the group-comparison contrasts (for task vs. baseline, gaze vs. gender, and two modulation contrasts), we extracted beta estimates (the first eigenvariate) within an 8mm radius sphere from the peaks of significant clusters, which were used for subsequent analyses. Example variable distributions, sorted by group, are displayed in Figures S2 and S3.

Latent Variable Analysis

To test associations among diagnosis, gaze perception (i.e., width and threshold), and social cognition, we used Bayesian latent variable modeling, implemented with MPLUS (Muthén & Muthén, 2017); Bayesian estimation of latent variable models provides optimal fit and is ideal for relatively small samples (Muthén & Asparouhov, 2012). A social cognition latent variable was indicated by MSCEIT branch scores, RME accuracy, and ER-40 accuracy; we also included an orthogonal methods factor for the MSCEIT. Our decision to use a single latent variable for social cognition was consistent with the results of a Velicer’s MAP test, which achieved a minimum value with one factor (MAP₁ = .07). We examined correlations among social cognition, diagnosis (HC = 0, SZ = 1), and gaze-perception metrics.

To test whether patterns of brain activity engaged during gaze perception were associated with gaze-perception performance, social cognition, and diagnostic group we used latent variable analysis with full-information robust weighted least squares estimation (WLSMV). We fit exploratory neural factors (using an oblimin rotation) from our extracted beta estimates for significant clusters from whole-brain analyses, which were then used as statistical predictors of the social cognition latent variable, width and threshold metrics from the gaze task, and diagnostic group (HC = 0, SZ = 1). The number of neural factors was selected based on the results of a Velicer’s minimum average partial (MAP) test (O’connor, 2000).

Following our primary latent variable analyses in MPLUS, factor scores for latent variables were computed and exported for follow-up moderation analyses and visualizations. To examine whether associations among gaze perception, social cognition, and neural factors differed between groups, we conducted follow-up regression models that included diagnosis and neural-factor-by-diagnosis interaction terms. We visualized key findings using scatter plots.

Finally, sensitivity analyses were conducted. First, we examined whether results changed when sex, age, cognitive ability (i.e., participants’ average of working memory and reasoning MATRICS battery t-scores), and data-collection subsample were included as covariates and when task-performance outliers were removed from the dataset. Outlier datapoints were identified using Rosner’s generalized ESD test (Rosner, 1983) and subsequently removed to form a new dataset for these sensitivity analyses. Finally, at the suggestion of reviewers, we estimated additional models including only ER-40, RME, and the perceiving emotions branch score from the MSCEIT as indicators of social cognition.

Associations with Clinical Symptoms

Lastly, we examined associations of symptom severity with our gaze perception metrics, social cognition, pre-/post-central gyrus activation, and neural factors. Given that a portion of our participants completed the SAPS/SANS while others completed the PANSS, we converted scores on the SAPS and SANS to equivalent scores on the PANSS using the procedure outlined by Grot et al. (2021). Then, correlations were examined for positive, negative, and total symptoms. We examined zero-order correlations of symptoms with gaze perception, social cognition, and brain activation. Finally, we conducted additional analyses controlling for multicollinearity among brain-activation metrics.

Transparency and Openness

Legally, our data cannot be made publicly available online, due to the wording of our IRB protocol and informed consent document. Study materials, data, and analytic code will be made available to readers upon reasonable request, by contacting S.D.B. or I.F.T. Data were analyzed using R (R Core Team, 2020) and MPLUS (Muthén & Muthén, 2017). The study was not preregistered.

Results

Behavioral Data

In the gaze task, SZ showed higher width and lower threshold, suggesting lower perceptual precision and higher self-referential bias. SZ also showed worse performance on all six of six observed social cognition variables. Means, standard deviations, and group differences are reported in Table S1 and variable distributions are presented in Figure S1. No behavioral or neural variables of interest were significantly correlated with SZ participants’ CPZ dose equivalents and no variables significantly differed between those with vs. without valid fMRI data.

fMRI Data

Second-level analyses across all participants revealed several significant clusters for each contrast (all-trials vs. baseline, gaze vs. gender, modulation by perception, and modulation by gaze angle). Clusters were distributed throughout regions canonically associated with social cognition, salience attribution, visual processing, and sensorimotor function (Figures S4–S9, Table S2). For the gaze vs. gender and modulation contrasts, no clusters significantly differed between HC and SZ; for the task-vs-baseline contrast, SZ showed greater activation in a cluster centered on left pre-/post-central gyrus (Figure 2, Figures S10–S15, Table S3). Moreover, in SZ, task-vs-baseline activation within this cluster was correlated with better perceptual precision (indicated by a negative correlation with width) on the gaze task (r = −.269, p = .040), but not self-referential bias (r = .069, p = .604) or latent social cognition (r = −.051, p = .696). Activation in the cluster was not associated with these variables in HC (p’s > .05). These results are visualized in Figure 2 and were consistent with follow-up moderation analysis that showed a significant interaction between group and pre-/post-central gyrus activation in predicting perceptual precision (β = −.202, p = .008).

Figure 2. Sensorimotor Hyperactivation and Perceptual Precision in SZ.

Note. SZ participants showed hyperactivation (a) within left pre-/post-central gyrus (b), which was associated with greater perceptual precision on the gaze task (c) and fewer symptoms (d). Findings are consistent with a moderation analysis that showed a significant interaction between group and pre-/post-central activation predicting perceptual precision (β = −.202, p = .008). *p < .05.

Latent Variable Analysis

Model for Gaze Perception, Social Cognition, and Diagnosis

Our behavioral latent variable model showed excellent fit (RMSEA = .015, 95% CI: [.000, .081], CFI = .997, 95% CI: [.936, 1.000], and TLI = .995, 95% CI: [.888, 1.000]). Results are visualized in Figure 3. In our measurement model of social cognition, all observed variables significantly loaded onto the latent factor. Better social cognition was associated with higher perceptual precision (i.e., a lower width parameter) but not with self-referential bias (i.e., threshold) on the gaze task; the strength of associations did not significantly differ between groups. Associations are visualized in Figure 4, using scatterplots of gaze-perception metrics and estimated social cognition factor scores. As in our t-tests reported above, SZ showed lower perceptual precision, higher self-referential bias, and worse social cognition.

Figure 3. Behavioral Model of Diagnosis, Gaze Perception, and Latent Social Cognition.

Note. MSCEIT = Mayer-Salovey-Caruso Emotional Intelligence Test, RME = Reading the Mind in the Eyes Test, ER-40 = Penn Emotion Recognition Test. The model depicts associations among diagnostic group, gaze perception metrics, and social cognition. Factor loadings for a social cognition latent factor (SCog) are shown on the right, as well as a method factor for the MSCEIT branch scores. Better social cognition was associated with greater perceptual precision. SZ showed worse social cognition, lower perceptual precision, and greater self-referential bias. *p < .05.

Figure 4. Scatter Plots of Gaze Perception Metrics and Social Cognition.

Note. Across both diagnostic groups, a lower width parameter on the gaze task (i.e., greater perceptual precision) was associated with better social cognition (r = −.343, p = .004). The threshold parameter (i.e., lower self-referential bias), however, was not significantly associated with social cognition (r = .194, p = .084). Diagnosis did not significantly moderate the associations of social cognition with width (β −.030, p = .678) or threshold (β = .076, p = .278). *p < .05, **p < .001.

In general, associations among gaze perception metrics, social cognition, and diagnosis were similar if we controlled for sex, age, cognitive ability, and dataset subsample or if task-performance outliers were removed from the dataset (For details see Figures S16–S17 and Table S6). A final analysis modeled social cognition using only the ER-40, RME, and the perceiving emotions branch from the MSCEIT (Figure S18); notably, associations between the gaze perception metrics and this reduced social cognition latent variable were stronger than those in our primary model (r = .421 for threshold and r = −.657 for width). This implies that the association between social cognition and gaze perception performance was driven largely by perceptual-based tasks.

Measurement Models for Neural Latent Variables

Our neurobehavioral latent variable model showed good fit (RMSEA = .022, 95% CI: [.012, .030]), CFI = .930, and TLI = .918). In terms of neural variables, six exploratory factors were extracted, based on the results of Velicer’s MAP test (MAP₆ = .017). See Table S4 for factors loadings. Neural factors correspond to 1) global task activation vs. baseline, 2) preferential activation for gaze (vs. gender), 3) preferential activation for gender (vs. gaze), 4) modulation (increase) of activation by perception of stimuli as self-directed and by viewing stimuli with objectively more self-directed gaze angles, 5) modulation (increase) of activation by viewing objectively more averted gaze angles, and 6) modulation (increase) of activation by perception of stimuli as averted. Note that voxel clusters that were more active in response to self-directed gaze—including those from the fMRI contrast for participant perception/behavioral endorsement and the contrast for objective gaze angle—loaded onto factor 4, whereas voxel clusters that were more active in response to averted gaze were split across factors 5 (for the objective gaze angle contrast) and 6 (for the participant perception/behavioral endorsement contrast). To facilitate interpretation, we choose to reverse signs when reporting associations of factors 3, 5, and 6 with diagnosis, social cognition, and gaze perception, given that these factors were indicated by variables with negative mean values for the whole-sample, second-level fMRI contrasts. (Thus, more-negative values, before signs were reversed, would represent stronger activation in the group-mean direction.)

Associations of Neural Factors, Social Cognition, Gaze Perception, and Diagnostic Group

Path coefficients from neural variables to gaze-perception metrics, social cognition, and diagnosis are shown in Figure 5. Three out of six neural factors were significantly associated with social cognition. Specifically, better social cognition was associated with stronger activation across neural factors marking global task activation vs. baseline (Factor 1), preferential activation for gaze vs. gender (Factor 2), and preferential activation for gender vs. gaze (Factor 3). Brain regions most strongly implicated in these neural factors included the following: fusiform gyrus, broad areas throughout PFC, hippocampus, and pre-/post-central gyri (Factor 1), insula, dmPFC, and IPL (Factor 2), and dlPFC, PCC/precuneus, angular gyrus, and STS/STG (Factor 3).

Figure 5. Whole-sample Regression Coefficients among Key Study Variables.

Note. The figure displays standardized path coefficients from each of the neural factors to criterion variables for diagnostic group, latent social cognition, gaze threshold (i.e., self-referential bias), and gaze width (i.e., perceptual precision). Neural factors correspond to 1) global task activation vs. baseline, 2) preferential activation for gaze (vs. gender), 3) preferential activation for gender (vs. gaze), 4) modulation (increase) of activation by perception of stimuli as self-directed and by viewing stimuli with objectively more self-directed gaze angles, 5) modulation (increase) of activation by viewing objectively more averted gaze angles, and 6) modulation (increase) of activation by perception of stimuli as averted. Voxel clusters that were more active in response to self-directed gaze—including those from the fMRI contrast for participant perception/behavioral endorsement and the contrast for objective gaze angle—loaded onto factor 4, whereas voxel clusters that were more active in response to averted gaze were split across factors 5 (for the objective gaze angle contrast) and 6 (for the participant perception/behavioral endorsement contrast). Significant positive associations are colored in red and negative correlations in blue. *p < .05, **p < .01.

In terms of neural factors and diagnostic group, only global task activation vs. baseline (Factor 1) showed a significant association; importantly, this association was negative (the opposite direction compared to its association with social cognition), suggesting patients showed broadly attenuated activation during both task conditions (i.e., gaze and gender) vs. baseline.

Reduced perceptual precision (i.e., a higher width parameter) was more specifically associated with reduced modulation of brain activity by viewing more direct vs. more averted gaze; this included factors spanning modulation of activation by both objective and subjective gaze self-directedness (Factors 4–6). Brain regions most strongly implicated in these neural factors included the following: insula, dmPFC, intraparietal sulcus, and orbitofrontal cortex (Factor 4); PCC/precuneus, pre-/post-central gyrus, STS/STG, and secondary visual cortex (Factor 5); fusiform gyrus, and precuneus (Factor 6). These findings suggest the psychophysical width parameter—an index of how strongly participants’ perceptual decision-making relates to changes in stimulus properties (i.e., gaze angle)—may track particularly well with modulation of neural activation by these same stimulus properties.

Patterns of association between neural factors and self-referential bias on the gaze task mirrored several of those seen for social cognition and perceptual precision. Specifically, stronger self-referential bias (i.e., a lower threshold parameter) was associated with less preferential activation for gender vs. gaze (Factor 3), as well as reduced modulation by viewing more direct vs. more averted gaze stimuli (Factors 4 and 5).

Follow-up Analyses

Associations were similar across the two diagnostic groups, as indicated by mostly nonsignificant interaction terms in follow-up regression analyses (Table S5). Significant interactions were, however, found between group and the neural factor marking task activation vs. baseline, when statistically predicting social cognition (β = .321, p < .001) and self-referential bias on the gaze task (β = .237, p = .034); interactions were positive, suggesting that the association between greater task activation and better social cognition (as well as lower self-referential bias) was particularly strong in SZ. Inspection of scatterplots and within-group correlations (Figure 6) suggested the association between social cognition and global task activation was driven by the SZ group and was not significant when examined in the HC group alone.

Figure 6. Plot of the Task Activation Neural Factor, Diagnostic Group, and Social Cognition.

Note. Diagnostic group was a significant moderator of the association between social cognition and the global task activation neural factor (β = .254, p < .001); the positive association observed in the whole sample was largely driven by an association within the SZ group. *p < .05, **p < .01.

In general, associations among gaze perception metrics, social cognition, neural factors, and diagnosis were similar if we controlled for sex, age, cognitive ability, and dataset subsample or if task-performance outliers were removed from the dataset. A final analysis modeled social cognition using only the ER-40, RME, and the perceiving emotions branch from the MSCEIT; again, results were mostly similar to those from the primary analyses. For details see Figures S19–S21 and Tables S6–S8.

Associations with Clinical Symptoms

Several significant associations were found between PANSS scores and our variables of interest (Table 2). More severe symptoms were associated with worse social cognition and perceptual precision. More severe symptoms were also associated with decreased preferential activation for gender vs. gaze and less modulation of activation by averted gaze. Finally, greater activation in the pre-/post-central gyrus cluster that differed between HC and SZ was associated with lower symptom severity, mirroring the association of better perceptual precision with activation in this region. Across both task performance and neural variables, stronger associations were seen for negative and total symptoms compared to associations with positive symptoms.

Table 2.

Associations of Symptoms with Gaze Perception, Social Cognition, and Neural Activation

	PANSS Negative	PANSS Positive	PANSS Total
Task Performance
Gaze Width	.433**	.269*	.506**
Gaze Threshold	−.207†	−.124	−.239†
Social Cognition	−.236*	−.190†	−.306**
Neural Activation (Zero-Order)
Pre-/Post- Central Activation	−.173	−.134	−.224†
Task activation	−.123	−.071	−.140
Gaze > Gender	.029	−.063	−.021
Gender > Gaze	−.218†	−.117	−.242*
Direct gaze	−.148	−.062	−.153
Objectively averted gaze	−.213†	−.141	−.255*
Perceived averted gaze	−.330**	−.040	−.272*
Neural Activation (Partial)
Pre-/Post- Central Activation	−.191	−.136	−.239*
Task activation	−.018	−.069	−.062
Gaze > Gender	−.096	−.184	−.199
Gender > Gaze	−.244†	−.201	−.323*
Direct gaze	−.166	−.036	−.151
Objectively averted gaze	−.044	−.105	−.106
Perceived averted gaze	−.335**	−.021	−.267*

Note.

^†p < .10,

^*p < .05,

^**p < .01

Discussion

In this study, we combined fMRI, psychophysics, social-cognition tasks, and latent variable analysis to elucidate whether gaze perception and associated brain activity are correlates of social cognition. We found that both behavioral and neural indicators of gaze perception were associated with individual differences in social cognition, across SZ and HC, and that these neurobehavioral indicators were particularly related to negative symptom severity. We also found broadly attenuated task-related activation—but enhanced sensorimotor activation—in SZ. Strength of this sensorimotor hyperactivation was associated with better perceptual precision on the gaze task and decreased symptom severity.

Reinforcing and Extending Previous Social Cognition Research

Behavioral results from the current study reinforce previous research on social cognitive deficits in SZ. We replicate findings of reduced perceptual precision and increased self-referential bias during gaze perception in SZ (Tso et al., 2012; Tso, Taylor, et al., 2021; Yao et al., 2018), as well as poorer social cognition as measured with the MSCEIT, RME, and ER-40 (Pinkham et al., 2018; Tso et al., 2014a). The significant association between latent social cognition and diagnostic group suggests that fairly broad—rather than task-specific—social cognitive deficits are characteristic of SZ.

The fact that social cognition and perceptual precision were associated not only with diagnostic group but also with symptom severity is relevant to explaining heterogeneity within SZ. Social cognition and gaze perception were most strongly related to negative symptoms, suggesting features such as anhedonia, avolition, and withdrawal may contribute to social cognitive dysfunction and visa-versa. This is consistent with several other studies that suggest negative symptoms are most strongly related to social cognition and related functional outcomes (Abplanalp et al., 2022; Burton et al., 2019; Strassnig et al., 2015; Ventura et al., 2009).

Our fMRI analyses suggest that gaze perception—as well as its modulation by stimulus properties and perception—involves multiple, widely distributed brain regions (e.g., insula, dmPFC, IPL/angular gyrus, PCC, TPJ/STS, precentral/postcentral gyri, and visual cortex); many of these regions have been previously discussed as having roles in the visual, salience, and default mode networks (Uddin et al., 2019). Parametric modulation of brain activation during the gaze task (throughout clusters in insula, dmPFC, STS/TPJ, and IPL) was correlated with a psychophysical index of perceptual precision for task performance, suggesting a close mapping between behavioral and neural response to salient features of gaze. This is consistent with the known roles of these brain regions in mechanisms critical to processing self-related visual information: for instance, the anterior insula for salience attribution, IPL for visuospatial processing, STS/TPJ for perception of social cues, and dmPFC for self-referential processing (Boyarskaya et al., 2015; Itier & Batty, 2009; Schilbach et al., 2006; Schobert et al., 2018; Uddin, 2015).

Because the processes involved in gaze perception are also fundamental to social cognition, it is fitting that we also found behavioral and neural indices of gaze perception were associated with participants’ performance across social-cognition tasks. These results add to a growing body of work linking social cognitive abilities to individual differences in structure and function of the dmPFC, insula, IPL, and TPJ/STS (Allen et al., 2017; Eres et al., 2015; Hou et al., 2017; Meda et al., 2014; Tso, Angstadt, et al., 2021; Udochi et al., 2022), extending past research through the use of multiple behavioral tasks, latent variable modeling, and a relatively large sample spanning patients and controls. The current findings are particularly novel and of note, as we show that gaze perception—a basic perceptual building block for complex social cognition—is related to various social cognitive abilities, spanning emotion perception, perceptual theory of mind, and emotion regulation.

Diminished Global but Enhanced Sensorimotor Activation in SZ

In our latent variable analysis, a global task activation neural factor was lower in SZ and associated with worse social cognition within the SZ group. This could be interpreted to suggest that broad patterns of task-related hypoactivation—rather than specific differences related to the more nuanced aspects of gaze perception indexed by our other contrasts—might underly many of the social cognitive deficits seen in SZ. This interpretation is consistent with work showing hypoactivation across the precuneus, insula, and prefrontal cortex in SZ for a variety of general and social cognitive tasks (Green et al., 2015; Kohler et al., 2008; Soldevila-Matías et al., 2022). In contrast, more specific patterns of neural activation that did not show group differences or interactions in the current study—such as those underlying the processing of gaze vs. gender and direct vs. averted gaze—may underly individual differences in social cognition that are not specific to SZ.

Contrary to the diminished global activation in SZ, across both task conditions vs. baseline, the SZ group showed enhanced activation within a cluster centered on left precentral/postcentral gyri—spanning motor, premotor, and somatosensory cortices. Counter-intuitively, this “abnormal” finding was also correlated with better gaze perception—higher perceptual precision—suggesting that it may be a compensatory mechanism. This is consistent with our finding that activation in this cluster was also associated with lower levels of symptom severity. In other words, some individuals with SZ may successfully engage in sensorimotor-driven processes to make up for broader patterns of diminished neural activation (such as those captured in our global task activation factor) that may negatively impact gaze perception.

Sensorimotor hyperactivation could be helpful to gaze processing in several ways. For instance, disambiguating gaze direction may be challenging to SZ participants due to dysfunction of regions such as the pMFC, insula, and TPJ/STS, and increased sensorimotor engagement may increase processing of stimuli and control of motor output, helping prevent patients from misinterpreting averted gaze as self-directed. Another possibility is that viewing various levels of averted gaze automatically triggers attentional shifts, with or without eye movements, toward the direction of gaze (Frischen et al., 2007); this is consistent with a meta-analytic finding that gaze perception shows more similarities in functional neuroanatomy with reflexive than with voluntary shifts of attention (Grosbras et al., 2005). Since shifting eyes or attention to the gazed-at location of the viewed face is inappropriate in the context of the gaze task, it requires inhibition or prompt disengagement from the gazed-at location to do well on the task. Prior studies have shown intact automatic attentional orienting to gaze (Langdon et al., 2017; Schwartz et al., 2010) but difficulties disengaging from a gazed-at location once shared attention is established in SZ (Langdon et al., 2017). Therefore, over-recruitment of the sensorimotor network may help overcome this difficulty and redirect attention to the task. Future studies could test whether sensorimotor over-recruitment is truly a compensatory neural mechanism in SZ by experimentally manipulating activation of this region in SZ (e.g., using brain stimulation techniques) and assessing the impact on social cognition.

Interpretations and Explanations of Null Effects

Contrary to our hypotheses, there were no significant diagnostic group differences for brain activation during the gaze vs. gender conditions or for modulation of activation by viewing direct vs. averted gaze. These null findings at least partially contradict previous work showing group differences in neural activity during gaze perception (Kohler et al., 2008; Pinkham et al., 2011) and a larger body of work on the neural bases of social cognitive deficits in SZ (Green et al., 2015). Nonetheless, the current study utilized a comparatively large sample and is consistent with several other studies that did not detect significant group differences in regional brain activations, despite significant associations between brain function and dimensional measures of social functioning (Abram et al., 2017; Fox et al., 2017; Horan et al., 2014; Oliver et al., 2021).

It is possible our task design contributed to the lack of group differences. Although our Gaze – Gender contrast reveals brain activation specific to explicit gaze perception, it is by no means a complete account of all processes involved in gaze perception. This is because gender identification still entails face processing and implicit gaze perception. Thus, some signals that are inherently part of gaze perception were likely subtracted away in the Gaze – Gender results. For example, basic visual processing deficits are well documented in SZ (Silverstein & Keane, 2011; Tso et al., 2014a) and may affect face processing during both explicit (i.e., distinguishing self-directed from averted gaze) and implicit (i.e., gender identification) gaze perception; likewise, both conditions may involve processes broadly related to social cognition and face perception. Unfortunately, these possibilities cannot be revealed with the current task design and will need to be explored in future investigations, which could use a control condition that does not allow normal face processing (e.g., scrambled faces or faces with the eyes region covered).

Within-Group Heterogeneity and Related Future Directions

Whereas task design may be one factor contributing to our lack of group differences, effects may also be masked by within-group variability in social cognition. Although a majority of SZ patients show social deficits, the magnitude of such deficits varies widely among patients and may be related to symptom severity (Hajdúk et al., 2018). Indeed, in the current study, we identified neural factors that were significantly related to symptom severity within the SZ group, despite not showing significant SZ vs. HC differences. Future studies could more carefully probe the role of various symptom dimensions in social cognitive deficits and associated neural pathways.

In addition to there being considerable within-group variability in social cognition itself, even similar overt deficits may emerge from heterogenous neurobiological mechanisms and symptom profiles (i.e., equifinality) (Cicchetti & Rogosch, 1996). For instance, social cognitive deficits in SZ could result from any combination of affective flattening, hypermentalizing, or cognitive disorganization, each of which may stem from unique symptom profiles and neurobiology (Bliksted et al., 2016; Frith, 2004; Green et al., 2015; Madeira et al., 2016). Better understanding of these alternative pathways to overt social deficits likely requires more advanced computational models of social cognition to parse the constituent cognitive and neural components. Future research should also incorporate a broader range of social cognition tasks, given that our current indices of social cognition were limited to the MSCEIT, RME, and ER-40, all of which focus on emotional processing. Studies that assess higher-level social cognitive abilities such as empathy, mental state attribution, mentalizing, or sarcasm perception (Abell et al., 2000; Buck et al., 2017; Corcoran et al., 1995; Johannesen et al., 2018; Stiller & Dunbar, 2007), examine the factor structure of social cognition, and examine associations of gaze processing with multiple social cognition factors would allow for even stronger conclusions. For instance, given that we found stronger associations with gaze perception when our social cognition latent variable included only lower-level/perceptual indicators (see Figure S18 vs. Figure 3), it is plausible that associations with gaze perception would further differ in magnitude when using tasks that measure higher-level theory of mind or empathy.

Future work should also strive to elucidate finer-grained links among SZ symptomatology, neurocognitive mechanisms, and social functioning. These questions could be further probed by including participants at elevated risk for schizophrenia, such as first-degree relatives and those with high schizotypy; these groups also tend to show abnormal social cognition (Stuke et al., 2021; Wastler & Lenzenweger, 2018), and research in non-patient samples can help overcome limitations related to medication use in SZ. Meanwhile, it could also be useful to incorporate additional diagnoses and symptom dimensions in future work, as social cognitive deficits are also prominent in autism (Pantelis & Kennedy, 2017), social anxiety (Jun et al., 2013; Schulze et al., 2013), and personality disorders (Roepke et al., 2013; Winter et al., 2017). Such an approach would be consistent with frameworks such as the NIMH RDoC (Insel et al., 2010) and the Hierarchical Taxonomy of Psychopathology (HiTOP) (Kotov et al., 2017), which seek to understand psychopathology in terms of underlying mechanisms and dimensions rather than diagnosis. Finally, although our sample size was comparable to or larger than those used in several previous studies using latent variable modeling/multivariate analyses to investigate brain function in schizophrenia (Lincoln et al., 2020; Liu et al., 2021; Plis et al., 2014), future research with even larger samples should be undertaken to confirm results from the current study.

Conclusion

Findings add to a growing body of work characterizing the neural substrates of gaze perception and their relation to social cognition. Given that both neural and behavioral indices of gaze perception were associated with performance across a range of social-cognition tasks, it appears that gaze perception may represent a key perceptual building block for social cognitive processes spanning emotion recognition, perceptual theory of mind, and emotion regulation.

In terms of clinical implications, it is becoming increasingly apparent that social cognitive deficits are a key dimensional feature related to psychopathology and functional outcome. Individual differences in social cognition across clinical and normal-range functioning appear to be supported by a complex set of perceptual processes and associated brain regions acting in tandem, which influence social abilities and outcomes via mechanisms that do not always clearly map onto diagnostic categories such as SZ.