Abnormal Gesture Perception and Clinical High-Risk for Psychosis

Abstract

Individuals diagnosed with psychotic disorders exhibit abnormalities in the perception of expressive behaviors, which are linked to symptoms and visual information processing domains. Specifically, literature suggests these groups have difficulties perceiving gestures that accompany speech. While our understanding of gesture perception in psychotic disorders is growing, gesture perception abnormalities and clues about potential causes and consequences among individuals meeting criteria for a clinical high-risk (CHR) syndrome is limited. Presently, 29 individuals with a CHR syndrome and 32 healthy controls completed an eye-tracking gesture perception paradigm. In this task, participants viewed an actor using abstract and literal gestures while presenting a story and eye gaze data (eg, fixation counts and total fixation time) was collected. Furthermore, relationships between fixation variables and both symptoms (positive, negative, anxiety, and depression) and measures of visual information processing (working memory and attention) were examined. Findings revealed that the CHR group gazed at abstract gestures fewer times than the control group. When individuals in the CHR group did gaze at abstract gestures, on average, they spent significantly less time fixating compared to controls. Furthermore, reduced fixation (ie, count and time) was related to depression and slower response time on an attentional task. While a similar pattern of group differences in the same direction appeared for literal gestures, the effect was not significant. These data highlight the importance of integrating gesture perception abnormalities into vulnerability models of psychosis and inform the development of targeted treatments for social communicative deficits.

Introduction

Abnormalities in the perception of expressive behaviors are characteristic of psychotic disorders.^1–5 Gesture, a complex form of nonverbal communication involving movements of the body, hands, and head accompanying speech, is one of the most critical, but least understood, expressive channels. Research in this area is important as gesture perception involves the integration of several functions impacted in psychosis (including motor, cognitive, and semantic processes),^4,6,7 and is foundational component of effective communication. Recently, investigators have identified gesture perception abnormalities in psychosis.^5,8,9 However, the nature of gesture perception in individuals meeting criteria for a clinical high-risk (CHR) syndrome is more limited. Investigating gesture perception in this group can inform our understanding of processes that interfere with communication, psychosis pathophysiology, and the development of targeted interventions.

Recent years have seen a growing interest in understanding gesture abnormalities in individuals diagnosed with schizophrenia.^10–15 Of this work, up to 50% of those in this group have been found to exhibit gesture deficits,^5,16,17 such as abnormalities with gesture performance.^14,18,19 Furthermore, studies indicate individuals with schizophrenia have impairments in the perception and interpretation of gestures.^5,10,20,21 For example, findings suggest deficits in the interpretation of hand gestures,²⁰ abnormalities in the perception of gestures that accurately/inaccurately match speech (ie, gesture-speech mismatch),¹⁰ and impairments in the recognition of specific types of gestures.⁵ Relatedly, gesture perception deficits are indicated to be pronounced when individuals are perceiving abstract (eg, gesturing a cup with one’s hand to represent the word “idea”) as well as literal gestures (eg, outlining a drawing of a house to depict the word “house”), although evidence appears to suggest abstract gestures are particularly affected.^22–24 This may be due to the complex and figurative nature of abstract gestures. Given that brain regional networks associated with gesture and language are overlapping,²⁵ findings suggesting disturbances in language-based paradigms such as difficulty interpreting proverbs, metaphors, and humor may generalize to the perception of abstract gestures as well.²²

Gesture abnormalities and relations with symptoms are observed in studies of schizophrenia. For example, reduced use of gestures predict functional outcomes 6 months later and are related to motor abnormalities, as well as positive and negative symptoms.^5,8,10,16,19 While studies investigating gesture perception and symptoms are more limited, findings point towards links with positive symptoms in these groups,⁵ and formal thought disorder.¹⁰ Furthermore, in psychotic disorders, nonverbal communication abnormalities have been tied to anxious and depressive symptoms.²⁶ Additionally, nonverbal communication deficits such as gesture abnormalities are observed to similar degrees in studies comparing individuals with schizophrenia and depression²⁷ as well as anxiety-related disorders.²⁸

Gesture perception deficits are also associated with impairments in domains of visual information processing in schizophrenia.^5,29 Broadly, individuals with schizophrenia exhibit visual information processing impairments such as with working memory and attention.^30–32 Similarly, gesture abnormalities have been linked with working memory impairment.⁵ Additionally, relationships have been documented between gesture perception and attentional processes in healthy populations.³³ Given that visual information processing is critical for perception of nonverbal information,^33,34 understanding relationships between these domains and gesture perception may hint towards whether difficulties processing visual information could be underlying gesture perception abnormalities.

Gesture abnormalities are also observed among individuals with a CHR syndrome. The focus of gesture research has been predominantly on gesture performance utilizing methods such as self-report, clinical interviews, and video coding in this group.^34–36 Relatedly, subtle gesture performance abnormalities are evidenced in early developmental stages such as within the premorbid period³⁶ and among other risk categories, including schizotypal personality disorder.³⁷ Impairments in producing abstract gestures have been identified in CHR groups as well,³⁵ although whether abnormalities are present in perceiving a speaker’s use of abstract and literal gestures is unknown. Furthermore, there is evidence of links between gesture performance and both symptoms and visual information processing domains in individuals with a CHR syndrome.^34,38 Lastly, while associations between other expressive deficits (ie, facial expressions) and mood symptoms in CHR groups have been investigated,^39,40 no study has assessed these relationships with gesture perception. This is an informative area of investigation given that many CHR individuals endorse comorbid diagnoses (eg, depression, anxiety)^41,42 and findings can inform vulnerability models and treatments, particularly surrounding dual diagnosis and clinical heterogeneity.⁴³

A novel avenue to assess gesture perception involves the use of eye-tracking paradigms, which allows for the precision in pinpointing visual attention. Eye-tracking systems can project an infrared light over participants eyes and calculates gaze position using the reflections of light from the front of the cornea and the back of the lens. Eye-tracking gesture perception studies are common in healthy populations³³ but lacking in those with psychotic disorders and individuals with a CHR syndrome. Furthermore, eye tracking has been used as a means to assess engagement/disengagement in individuals with depression.⁴⁴ Eye tracking is a useful tool to assess gesture perception and specifically, whether individuals are visually attending to information and for how long.

In the present study, gesture perception was assessed in a sample of individuals meeting criteria for a CHR syndrome when compared to controls. Specifically, participants completed an eye-tracking paradigm in which they were asked to view a video of an actor using abstract and literal gestures while describing a situation (ie, a day at the park). During viewing, fixation data (ie, eye gaze) was collected using an eye tracker in order to determine the number and amount of time participants spent fixating on each gesture type. Given gesture perception abnormalities have been documented in studies with schizophrenia,^4,5,17 and this has been particularly the case in the perception of abstract gestures,^22–24 we predicted that the CHR group would not fixate on abstract gestures as much as controls. Furthermore, during fixation, CHR group would spend less time gazing at abstract gestures also when compared to controls. Next, relationships between abnormalities in gesture perception and symptoms (eg, positive, negative, disorganized communication, anxiety, and depression), as well as measures of visual information processing commonly observed in both the psychosis⁵ and healthy literature³³ (ie, working memory and attention) were examined. Based on these studies and the broader literature, we predicted that abnormalities in eye-tracking metrics would be related to symptom severity and impaired visual information processing.

Methods

Participants

A total of 61 participants, aged 15–24 (M = 20.30, SD = 2.25), including 29 CHR individuals and 32 healthy controls were recruited to the Adolescent Development and Preventive Treatment (ADAPT) Program via website advertisements, flyers, and community referrals, and the Northwestern University Psychology recruitment pool. Exclusion criteria for both groups included a current or past head injury, neurological disorder, substance dependence, IQ < 70, and diagnosis of a schizophrenia spectrum disorder or mood disorder with psychosis. Additional exclusion for controls included having a first-degree relative with a psychotic disorder.

A CHR syndrome was given based off of ratings from based off of ratings from the Structured Interview for Psychosis-Risk Symptoms (SIPS).^45,46 For those meeting CHR criteria, individuals endorsed either attenuated positive symptom syndrome (APSS), receiving a rating of a 3 (moderate) to a 5 (moderately severe) on the attenuated positive symptom domain. Furthermore, individuals met CHR criteria if they had a first-degree relative diagnosed with a psychotic disorder and/or schizotypal personality disorder accompanying a decline in global functioning. The Structured Clinical Interview for the DSM (SCID)⁴⁷ was used to rule out psychosis. All interviewers were extensively trained on clinical interviews and were reliable (κ > .80). All participants consented and the research was approved by Northwestern University’s Institutional Review Board.

Measures

Eye Tracking Paradigm: “A Day at the Park”.

Participant fixation data (ie, eye gaze) was collected using Tobii TX-120 eye tracker system (figure 1). Participants sat approximately 60 cm from the eye tracking device and viewed a monitor that was 35 × 20 inches, with a resolution of 1920 × 1080. To begin with, participants underwent a brief calibration phase in which they viewed a red dot that moved along different paths along the screen (eg, horizontally, diagonally). After calibration, participants viewed a 2-minute video of an actor that verbally described a day at the park while producing simultaneous abstract and literal gestures. The actor was faced forward and eye gaze was forward as well. The actor remained stationary except for when gesturing (hand gestures were solely assessed in the current study, but the actor utilized gestures such as smiling, eyebrow raises, and shoulder shrugs to ensure the story description was reflective of natural communication). Examples of abstract gestures from the video include when the actor placed their hands on their heart to depict the word “favorite.” Examples of literal gestures from the video include putting 6 fingers up to represent the word “six.” Furthermore, basic factual and conceptual questions were asked about the story after the experiment to ensure engagement and there were no group differences in the number of correct answers (P > .40). After data collection was complete, areas of interest (AOIs) were set throughout the video for each gesture using Tobii Studio Version 2.3.1. Final variables of interest include (1) the number of times not fixating (scores of 0) on gesture perception categories and (2) total amount of time individuals fixated on abstract and literal gestures. The number of times not fixating (termed fixation count throughout) provides information as to whether individuals are looking at the gestures, while fixation time provides information as to if they are looking at the gesture, for how long.

Fig. 1.

(A) An example of a gesture depicted in the video and (B) fixation points from the eye-tracking task. Note. (A) An example of a gesture depicted in the video while actor is saying “6 months ago”; (B) The cluster depicts where individuals were fixating during viewing, with 93% participants gazing within the shown cluster at this point in the video.

Symptoms.

Positive and negative symptom domain sum scores and a disorganized communication (disorganized thinking reflected through speech) item were used from the SIPS interview. Furthermore, measures of depression and anxiety were obtained using the Beck Depression Inventory (BDI)⁴⁸ and Beck Anxiety Inventory (BAI),⁴⁹ respectively.

Working Memory and Visual Attention.

The Penn Letter N-Back Test^50,51 was administered to measure working memory. In this task, numbers appeared on the screen one at a time at a rate of 0.50 seconds. Participants were instructed to press the spacebar when they saw an “X” (0-back condition), when 2 letters were presented consecutively (1-back condition), and when 3 letters were presented in a row (2-back condition). Variables of interest were the number (accuracy) and response time of correct responses (speed).

The Penn Continuous Performance Task—Number Letter Version^50,52 was administered to test vigilance and visual attention. In this task, lines (vertical, horizontal) appeared on the screen at a rate of 1 second. Participants were instructed to press a key when the lines formed a number or letter. Variables of interest were true positives (accuracy) and median response time of correct responses (response time/attentional speed).

Data Analysis.

Independent and chi-square tests were used to assess for demographic characteristics. Tests of normality suggested that fixation counts and fixation time on abstract gestures variables were normally distributed, and thus, parametric tests were used (ie, independent t-tests, Pearson correlations). For the literal gesture perception category, normality tests indicated the variable was not normally distributed and thus, non-parametric tests were used in subsequent analyses (ie, Mann-Whitney U, Spearman correlations). Bivariate correlations were applied to investigate relationships between gesture perception categories and (1) symptoms (positive, negative, disorganized, depression, and anxiety) and (2) visual information processing domains (ie, working memory and attention). Please note analyses are not corrected for multiple comparisons.

Results

Demographics

There were no significant group differences in age or parental education. There was a significant group difference in the distribution of biological sex in that the control sample had more females than the CHR sample. As a result, we examined the effect of sex in the following section. As expected, the CHR group reported more positive symptom severity than the control group, with a similar pattern for negative symptoms and disorganized communication as well as depressive and anxious symptoms. There were no differences in cognitive variables except for working memory accuracy (table 1).

Table 1.

Means and Standard Deviations for Demographics, Symptoms, and Visual Information Processing Domains

	CHR	Control	Total	Statistic
Demographics
Number of participants	29	32	61
Age	20.83 (2.39)	19.81 (2.02)	20.30 (2.25)	t(58) = 1.796, P = .08
Biological Sex (% female)	41%	66%	54%	x2 (1) = 4.21, P = .04
Parent Education (years)	16.70 (2.05)	15.38 (3.14)	15.98 (2.78)	t(56) = 1.88, P = .07
Race and Ethnicity
Asian/Middle Eastern	10%	19%	15%
African-American	31%	9%	20%
Central/South American	3%	0%	2%
White	45%	56%	51%
Interracial	11%	13%	11%
Unknown	0%	3%	1%
Symptoms
Positive	12.52 (3.47)	0.31 (0.60)	8.18 (6.53)	t(43) = 18.44, P < .001
Negative	9.07 (5.56)	0.81 (1.76)	6.13 (6.06)	t(43) = 7.36, P < .001
Disorganized Comm	1.96 (1.04)	0.19 (0.54)	1.32 (1.23)	t(42) = 7.46, P < .001
Depression	16.96 (12.75)	5.87 (5.26)	11.14 (11.01)	t(57) = 4.29, P < .001
Anxiety	17.71 (14.41)	8.52 (8.21)	13.20 (12.56)	t(53) = 2.92, P = .006
Visual Information Processing
Visual Attention Accuracy	112.63 (10.21)	113.71 (11.77)	113.18 (10.94)	t(53) = −0.37, P = .72
Visual Attention Speed (ms)	479.13 (30.58)	477.48 (45.20)	478.29 (38.37)	t(53) = 0.16, P = .88
Working Memory Accuracy	26.70 (2.97)	29.07 (0.90)	27.91 (2.47)	t(53) = −0.03, P < .001
Working Memory Speed (ms)	545.68 (113.11)	528.91 (99.79)	537.14 (105.88)	t(53) = 0.58, P = .56

Note: Parental education is the average of mother and father education in years; Positive and negative symptoms are sum scores taken from the Structured Interview for Psychosis-Risk Syndromes (SIPS); Disorganized communication (“Disorganized Comm”) is an item from the SIPS positive symptom domain; Depression and anxiety are sum scores deduced from the Beck Depression Inventory and Beck Anxiety Inventory, respectively; Visual informational processing domains are extracted from the Computerized Neurocognitive Battery (CNB); Visual Attention Accuracy is obtained from the Penn Continuous Performance Test—Letter Number Version and represents the number of true positives as the accuracy store; The median response time of true positives is represented by Visual Attention Speed; Working memory scores are taken from the CNB Letter N-Back; Working Memory Accuracy is the total number of correct responses, while working memory speed represents the median response time (speed); ms = milliseconds.

Group Differences in Gesture Perception Categories

The CHR group had less fixation counts on abstract gestures compared to controls, t(59) = 2.56, P = .013, d = 0.66 (figure 2). In addition, when participants were gazing at the abstract gestures, they were also spending significantly less time fixating on abstract gestures also compared to controls, t(59) = −2.26, P = .028, d = 0.47. There were no significant differences between groups in fixation counts for literal gestures, U = 425, P = .57, or when there was fixation, the amount of time they were fixating on literal gestures, U = 423, P = .55.

Fig. 2.

Group differences between a sample of individuals with a clinical high-risk syndrome and controls in (A) fixation counts and (B) fixation time for abstract and literal gesture perception. Note. *P < .05. Error bars represent standard error.

Relationships Between Gesture Perception and Symptoms Within the CHR Group

Less fixation counts (more times not looking) on abstract gestures was related to anxiety severity at a trend level, r = .37, P = .055, but not positive, negative, disorganized communication, or depressive symptoms (P > .13). However, less time fixating on abstract gestures was significantly related to depression severity, r = −.41, P = .03 (figure 3). In terms of relationships with anxiety, less time gazing at abstract gestures was marginally related to anxiety, r = −.35, P = .07. There were no significant relationships between fixation time on abstract gestures and positive or negative symptoms (P > .37) as well as disorganized communication (P = .91). In terms of relationships between literal gesture perception and symptoms, there were no significant associations (P > .06).

Fig. 3.

Associations between fixation time and (A) depression and (B) response time in a clinical high-risk sample. Note. (A) Beck Depression Inventory sum scores; (B) The median response time of true positives is represented by response time; ms = milliseconds.

Relationships Between Gesture Perception and Domains of Visual Information Processing Within the CHR Group

Less fixation counts (more times not looking at the gestures) on abstract gestures were not related to accuracy of responses on the attentional task, r = .20, P = .32, but was significantly related to slower response time, r = .46, P = .015. Additionally, there were no significant relationships between fewer fixation counts and working memory accuracy or time (P = .97). Similarly, less fixation time on abstract gestures was not significantly related to accuracy of responses on the test of attention as well, r = −.24, P = .22, but was associated with slower time for correct responses, r = −.42, P = .029. Furthermore, performance on the test of working memory revealed there were no significant associations between abstract gesture fixation time and the number of correct responses, r = −.17, P = .93, or response time for correct items, r = −.13, P = .53. There were no significant relationships between perception of literal gestures and visual information processing domains (P > .12) except less fixation counts were significantly related to slower response time, r = .52, P = .005.

Exploratory Analyses Investigating Sex Differences in Gesture Perception

An interaction (group by sex) predicting eye tracking metrics was examined (see supplementary table S1 for demographic details). Findings revealed no significant interaction in predicting fixation counts for abstract, F(56) = 0.63, P = .43, or literal gestures, F(56) = 0.04, P = .84. Similarly, no significant interaction was observed when predicting fixation time for abstract gestures, F(56) = 0.53, P = .22, or literal gestures, F(56) = 0.32, P = .58. Despite null interaction effects, group differences between CHR/control females and males were assessed (supplementary table S2) as well as correlations between eye tracking metrics and symptoms (supplementary table S3) and visual information processing domains (supplementary table S4).

Discussion

The current study assessed gesture perception using an eye-tracking paradigm and relationships between symptoms and visual information processing domains in a sample of individuals meeting criteria for a CHR syndrome when compared to controls. Findings revealed that the CHR group, on average, did not attend to the abstract gestures as frequently as controls indicating that gesture perception abnormalities may not be solely a visual information processing deficit. Additionally, when this group did fixate on gestures, they spent significantly less time fixating on abstract gestures. Finally, less fixation time on abstract gestures was related to depression and slower response times within the CHR group. Surprisingly, we did also see a pattern in which those with a CHR syndrome were also not fixating on literal gestures, although the effects were not significant. These data shed light on our understanding of gesture perception abnormalities in this group and stress the utility of using eye-tracking as a means for assessing gesture perception. Importantly, findings inform vulnerability models, prevention strategies, and targeted treatments for social-communicative deficits.

As mentioned, a central finding of this study was that the CHR group was not fixating as frequently on abstract gestures and when there was fixation, the time was less compared to controls. Furthermore, there was a pattern suggestive of reduced fixation on literal gestures in the CHR group compared to controls, although the effect was nonsignificant and it is possible this may be due to limited sample size. Interestingly, the eye-tracking method was able to pick up, with high sensitivity and precision, where exactly individuals were gazing in the video. These data offer additional support for the use of eye-tracking methods, as historically, eye tracking has not been commonly employed to study gesture in schizophrenia or CHR groups. Altogether, the current findings suggest abstract gesture perception is more pronounced compared to literal gesture perception in this sample of individuals with a CHR syndrome compared to controls. These data support evidence from studies of schizophrenia documenting abnormalities in gesture perception, particularly in regards to abstract gestures.^10,24 These data also coincide with work in schizophrenia, indicating individuals in these groups have difficulties understanding the meaning of proverbs and metaphors in verbal communications.²² As mentioned, language and gesture comprehension have similar underlying neural pathways^23,53–55 and perceiving gesture may serve as an additional tool to process verbal information.

Associations between eye-tracking metrics for abstract gestures and depression provide nuanced perspectives to the literature. As mentioned, comorbidity such as depression is quite high in CHR groups.^41,42 Leading theories on depression suggest that thoughts, attitudes, and interpretations can interfere with the way an individual with depression is able to attend to and recall information. Furthermore, increasing literature in this area suggests that individuals diagnosed with depressive disorders attend to negative information.⁵⁶ In this case, it is possible that depressive symptoms endorsed by CHR groups may underlie difficulties attending to complex, but relevant information co-occurring with speech such as abstract gestures. This difficulty may overlap with the notion that limited cognitive resources play a role in expressive deficits as well.⁵⁷ However, future work investigating CHR subgroups experiencing comorbid diagnoses and continuing to understand relationships between depression and gesture perception could shed light on the interpretation of these findings. Furthermore, it is important to note that no associations were observed between literal gesture perception and symptoms. This data further hints towards the possibility that difficulties perceiving abstract gestures, in particular, may be central to the CHR syndrome. However, more well-powered replication studies are needed before definitive conclusions can be made.

The results of this study also revealed no relationships between eye-tracking metrics for abstract gestures and the number of true positives on the attentional tasks. However, associations were found with slower response time. This is particularly striking when considering both groups performed relatively the same on these attentional tasks. In contrast, these findings are perhaps not unsurprising given that eye-tracking methods are a way to measure visual attention in particular. However, these data offer a new perspective in our understanding of gesture perception and the CHR syndrome. Specifically, findings suggest that those in this risk group may not be attending to abstract gestures. These data inform treatment interventions in that targeting social signals such as viewing gestures to interpret information may be of use. This study also suggests for instances in which individuals are fixating on gestures, it may be a result of difficulty with visual information processing. These data are in line with studies in schizophrenia and CHR groups highlighting relationships between gesture deficits and impaired visual information processing.^5,8,17,34 Furthermore, relationships between less fixation time on abstract gestures and slower attentional speed may be reflective of challenges with shifting attention quickly⁵⁸ although more research is warranted. While there is evidence of working memory deficits in schizophrenia and CHR groups as well,⁵⁹ there were no associations between eye-tracking metrics and working memory. These contrasting results may indicate that gesture perception abnormalities related to fixation time are a visual attention-related impairment rather than suggestive of difficulties processing and updating information, and this may be particularly the case with abstract gesture perception. It is also important to note similar findings suggesting more time not looking at literal gestures was related to slower response time. However, in light of null group differences, interpretations are difficult to make and should be done so with caution. Further research, with larger sample sizes, is needed to better understand relationships between literal gestures and cognitive functions.

While there are several strengths to the study, there are limitations to discuss as well. First, the study was a cross-sectional design and future work is needed with longitudinal designs. Furthermore, more research is needed to understand biological sex differences in the perception of abstract and literal gestures. In the current study, we focused our analyses on abstract and literal gestures; future studies should consider assessing other types of gestures (eg, beats). Furthermore, utilizing eye tracking as a means to assess other types of gesture abnormalities is a future direction as well; eye tracking is a tool that call allow for specificity of visual attentional processes. Furthermore, there may be important clues in investigating both the perception and interpretation of each gesture by probing as to whether individuals understood each gesture. Lastly, future research would benefit from investigating the effects of contextual information from the video (eg, body orientation, eye gaze, speech content of the actor).⁶⁰

Funding

This work was supported by the National Institute of Mental Health Grants: F31MH121018-01A1 to T.G., F31MH123121-01A1 to K.J.O., and 1R01MH112545-01, R21MH119677, R21MH110374, R21MH115231, R21/R33MH103231 to V.A.M.