Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Sep 1.
Published in final edited form as: Schizophr Res. 2023 Mar 17;259:20–27. doi: 10.1016/j.schres.2023.03.014

Linguistic correlates of suicidal ideation in youth at clinical high-risk for psychosis

Matthew F Dobbs a,*, Alessia McGowan a,*, Alexandria Selloni a, Zarina Bilgrami b, Cansu Sarac c, Matthew Cotter a, Shaynna N Herrera a, Guillermo A Cecchi d, Marianne Goodman a,e, Cheryl M Corcoran a,e, Agrima Srivastava a
PMCID: PMC10504409  NIHMSID: NIHMS1883679  PMID: 36933977

1. Introduction

Those with psychotic disorders, such as schizophrenia, are particularly vulnerable to suicidal ideation and behavior (SIB; Chang et al., 2014; Yates et al., 2019), and are roughly thirteen times more likely to die by suicide than the general population (Too et al., 2019). Unlike the general population, however, in which the risk for suicide increases through middle age (Hedegaard et al., 2021), the risk of suicide in psychotic disorders is highest during young adulthood (Nordentoft et al., 2004), around the onset of illness (Palmer et al., 2005; Upthegrove et al., 2010), with approximately half of psychosis-related suicide attempts occurring around the first episode of psychosis (Nielssen & Large, 2009). Furthermore, whereas approximately 1 in every 7418 people in the general population die by suicide (Center for Disease Control, 2020), 1 in 58 individuals diagnosed with schizophrenia spectrum disorders will die by suicide within the first four years of their diagnosis (Zaheer et al., 2020). The atypical timing as well as the increased rates of suicide in this clinical population underscore the importance of understanding this intersection of early psychosis and suicide risk.

SIB is also prevalent during the putative prodromal period of psychotic disorders, namely clinical high-risk (CHR) teens and young adults with subthreshold psychotic symptoms (Correll et al., 2010; Miller et al., 2003). The prevalence of death by suicide among CHR youths is 1.25%, suicidal behavior occurs in almost 20% of CHR youths, and lifetime suicidal ideation characterizes the majority of CHR individuals (Gill et al., 2015; Hutton et al., 2011; Taylor et al., 2015). While youth at CHR are clearly at an elevated risk for suicide, current metrics are not yet able to reliably detect relevant warning signs in real time. Studies of correlates of SIB in CHR individuals have primarily focused on symptoms and exposures, variably identifying depression, negative symptoms, psychotic-like symptoms (e.g., suspiciousness), substance abuse, and childhood trauma as correlates (Gill et al., 2015; Hutton et al., 2011; Taylor et al., 2015). In a prior study (DeVylder et al., 2012), we found prevalent suicidal ideation among CHR individuals – primarily a wish to be dead – that was associated with both depressive symptoms and obsessions of an aggressive nature. Of note, only a few of these studies have used the Columbia-Suicide Severity Rating Scale (C-SSRS; Posner et al., 2011) from the PhenXtoolkit, which further characterizes the intensity of suicidal ideation. Importantly, suicidal ideation has been identified as a precursor to an actual attempt in the suicide process (Forman et al., 2004; Park et al., 2020; Van Heeringen et al., 2000).

Another clue to SIB may lie in what people say or “talk” about (e.g., expressing feelings of hopeless, having no reason to live), which the American Foundation for Suicide Prevention identifies as a “warning sign” of suicidal behavior (American Foundation for Suicide Prevention, 2019). Computational approaches have been developed that can analyze the structure and content of language, known as natural language processing (NLP; Corcoran & Cecchi, 2020). NLP studies in cohorts other than CHR and psychosis have examined language across modalities, including poetry, social media posts, texts and to a more limited extent spoken language; these findings link SIB to using more words associated with “anger”, “sadness”, “loneliness” and “stress” (Agurto et al., 2018; Coppersmith et al., 2015; Glenn et al., 2020; Homan et al., 2022; Ji et al., 2018; Nguyen et al., 2017; Pestian et al., 2016; Ramírez-Cifuentes et al., 2021; Shah et al., 2019), as well as increased use of the personal pronoun “I” across various (Agurto et al., 2018; Coppersmith et al., 2015; Homan et al., 2022; Ji et al., 2018; Leavitt et al., 2021; Lekkas et al., 2021; O’Dea et al., 2021; Tadesse et al., 2020). While NLP has been applied to transcripts of natural spoken language in individuals with schizophrenia and CHR youths, characterizing decreases in semantic coherence (e.g., tangentiality) and syntactic complexity (e.g., poverty of speech) (Bedi et al., 2015; Corcoran et al., 2018), NLP has not been used yet to characterize suicidal ideation in psychosis and CHR, consistent with a recent systematic review (Homan et al., 2022).

Based on these previous findings, the goal of this study was to use NLP to identify linguistic correlates of recent suicidal ideation, characterized by the C-SSRS, in CHR individuals. We hypothesized that the content of spoken language in CHR individuals with recent suicidal ideation (vs. those without) would have greater semantic similarity to words like “anger”, “sadness”, “loneliness”, and “stress”. We also hypothesized that CHR individuals with recent suicidal ideation would have increased use of the personal pronoun “I”. We also used NLP in an exploratory manner to determine if there were patterns of speech associated with recent suicidal ideation in CHR youth that were not specifically hypothesized. To test this first hypothesis, we used the GoEmotions dataset (Demszky et al., 2020) and zero-shot text classification (Yin et al., 2019). To test this second hypothesis, we used “part-of-speech” tagging (POS-Tag) to estimate rates of usage of speech. We also used POS-Tag methods to determine if there were patterns of speech associated with suicidal ideation in CHR youth that were not specifically hypothesized.

2. Materials and Methods

2.1. Participants

Participants in this study were ascertained from a variety of sources, including clinical referrals, self-referrals, flyers, advertisements through social media, and website listings. Fluency in English was an inclusion criterion across all cohorts. All CHR participants were help-seeking and met criteria for a psychosis-risk syndrome, using the Structured Interview for Psychosis-Risk Syndromes/Scale of Psychosis-Risk Symptoms (SIPS/SOPS; Miller et al., 2003). Exclusion criteria included a history of threshold psychosis (determined by the Presence of Psychosis criteria on the SIPS/SOPS), any major neurological or medical disorder, IQ less than 70, and current risk for suicidal or violent behavior. Specific exclusion criteria for healthy controls included that individuals could not have any DSM-5 diagnoses other than a lifetime (not current) depressive disorder, as determined using the Structured Clinical Interview for DSM-5 Disorders (SCID-5; First et al., 2015). Written informed consent was obtained for all participants, with parental informed consent (and assent) provided for participants under the age of 18. This study was approved by the Institutional Review Board at the Icahn School of Medicine at Mount Sinai.

2.2. Measures

Data was collected in cross-section on demographics, diagnoses, clinical features, and recent suicidal ideation.

2.2.1. Diagnoses, Symptoms & Functioning

The SIPS/SOPS (Miller et al., 2003) was used to determine case status for CHR participants and was administered to all participants to assess symptoms (positive, negative, disorganized, and general symptoms), as well as a global assessment of function.

The SCID-5 (First et al., 2015), was used to determine other psychiatric diagnoses and was administered by licensed clinicians (psychologist, psychiatrist) who were trained and certified.

2.2.2. Suicidal Ideation & Behavior

The C-SSRS (Posner et al., 2011) is a comprehensive assessment that characterizes recent and lifetime suicidal ideation, including level (e.g., wish to be dead, with or without intent and/or plan), nature (of intent and plan, if present), and intensity (frequency, duration, controllability, potential deterrents, reasons), as well as the characterization of suicidal behaviors and attempts. The scale is designed to assess suicidal ideation and behavior in the past-month (recent) and over the entire life course (lifetime). The C-SSRS is included in the PhenX Toolkit for classification of suicidal ideation in adolescents and adults; to date, it has been used in more than 400 studies (PubMed). Each case was reviewed in a consensus meeting with an expert in the assessment of suicidal ideation (co-author MG). The CHR group was stratified in respect to those who endorsed suicidal ideation within the past month (CHR+rSI) versus those who did not (CHR−rSI). CHR individuals with and without suicidal ideation were compared across demographics, symptoms, and medications.

2.2.3. Language Sample Elicitation

Natural speech was elicited using narrative, open-ended interviews, which lasted approximately 30-minutes (Ben-David et al., 2014). Interviewers were trained in qualitative techniques designed to elicit spontaneous natural language. The first and only prompt by the interviewer was, “How have things been going for you lately?” The interviewer then followed the subject’s natural line of conversation, probing only to encourage the participant to further elaborate on his/her topics of choice. Interviews were audio recorded and audio files were transcribed by a HIPAA-compliant transcription service, TranscribeMe! (www.transcribeme.com).

2.3. Data Analyses: Natural Language Processing

Transcripts were preprocessed using the Natural Language Toolkit (NLTK; www.nltk.com), with removal of stop words and punctuations (commas, periods, etc.), conversion of sentences to lower case, lemmatization, and expansion of contractions. Preprocessing was followed by POS-Tag analysis, using the Penn Tree Bank (Santorini, 1990) in the NLTK, with the frequency of each POS-Tag calculated for the full interview. We estimated the rates of usage of the first-person singular pronoun “I” and, in exploratory analyses, used POS-Tag to assess rates of usage of the 36 parts of speech listed in the Penn Tree Bank.

To visualize the respective distributions for linguistic variables, we developed plots for their Cumulative Distribution Function (CDF), which is an integral of the histogram that shows a relative probability at a given value that is less than or equal to that value. CDFs are easy to interpret as values like minimum, maximum, median, quartiles, and percentiles can be discerned directly from the plot. In addition, CDFs also give a clear idea of outliers at the CDF tails. Kolmogorov-Smirnov tests were done to compare distributions for linguistic features between the study groups. Language analyses were adjusted for demographic features that different between the two subgroups.

We used the Bidirectional Encoder Representations from Transformers (BERT) base (Devlin et al., 2018) to train the GoEmotions dataset (Demszky et al., 2020) for the identification of emotion expression in language, with a focus on hypothesized emotions of “anger” and “sadness”. GoEmotions is a human-annotated dataset of approximately 58,000 Reddit comments extracted from popular English-language subreddits and labeled with 27 emotion categories. It includes twelve positive, eleven negative, and four ambiguous emotions, and one neutral, making it suitable for conversation-understanding tasks that require a subtle differentiation between emotion expressions. We used the BERT model to build a multi-label classifier that takes sentences and outputs the probability distribution for the emotions. Of note, GoEmotions-trained BERT-based models have been used to identify symptoms of depression and anxiety in therapy session transcripts (Burkhardt et al., 2022), outperforming Linguistic Inquiry and Word Count, an alternate software package, for the detection of linguistic indicators of mental state (Tausczik & Pennebaker, 2010), stress detection in online posts (Turcan et al., 2021), and prediction of depression, suicide and self-harm in social media (Tuarob et al., 2022).

As only two of the hypothesized emotions – “anger” and sadness” – are among the 27 emotions captured by the GoEmotions database, we also used zero-shot text classification (Yin et al., 2019) to determine the extent of semantic similarity to “loneliness” and “stress”. Zero-shot text classification is an Unsupervised Machine Learning technique that is used to classify text without using any labeled data (Yin et al., 2019). We additionally used zero-shot analysis for “anger” and “sadness” to determine robustness of results. Application of zero-shot modelling has been used to increase the utility of language samples that are often unstructured and unlabeled for emotional indicators (Liang et al., 2021). The zero-shot approach was recently used to examine open-ended interviews and dialogue in a large study of individuals with schizophrenia and autism spectrum disorder (ASD; Wawer et al., 2022), which revealed that training a Universal Sentence Encoder model from a schizophrenia language dataset provided ASD-predicting information. Further details regarding NLP methodology can be found in the supplemental materials.

3. Results

3.1. Sample Characteristics

Of the 43 CHR participants in this study, 10 endorsed suicidal ideation in the past month (CHR+rSI), whereas 33 did not (CHR−rSI). While there was a preponderance of females in the CHR+rSI group (60% vs. 40%), the CHR+rSI and CHR−rSI groups were similar in race, ethnicity, age, comorbid diagnoses (depression, anxiety, substance use disorder), positive and negative symptoms, and global functioning (Table 1). Use of antipsychotics was equivalent (approximately 20%) but there was a greater, albeit non-significant, use of antidepressants among CHR+rSI individuals. As use of emotions in language may differ by sex (Badal et al., 2020), and the CHR groups differed by sex, all further language analyses were adjusted for sex.

Table 1.

Demographics, Medications, and Clinical Characteristics

CHR+rSI CHR−rSI Controls
n=10 n=33 n=14
Age (years)
 Mean (SD) 24.1 (6.0) 22.8 (4.2) 25.5 (3.9)
 Min, Max 16.1, 32.3 16.3, 32.2 18.5, 33.9
Sex, n (%) *
 Female 6 (60.0) 12 (36.4) 9 (64.3)
 Male 4 (40.0) 21 (63.6) 4 (35.7)
Race, n (%)
 Black 2 (20.0) 5 (15.2) 2 (14.3)
 Asian/Pacific Islander/Hawaiian 3 (30.0) 8 (24.2) 6 (42.9)
 Other/More Than One Race 3 (30.0) 6 (18.2) 0 (0.0)
 White 2 (20.0) 11 (33.3) 6 (42.9)
 Did Not Disclose 0 (0.0) 3 (9.1) 0 (0.0)
Ethnicity, n (%)
 Hispanic/Latinx 4 (40.0) 12 (36.4) 1 (7.1)
 Not Hispanic/Latinx 6 (60.0) 21 (63.6) 13 (92.9)
Co-occurring Lifetime or Current
SCID Diagnoses, n (%)
 Depressive Disorder 6 (60.0) 16 (48.5) 3 (21.4)
 Anxiety Disorder 4 (40.0) 14 (42.4) 0 (0.0)
 Substance Use Disorder 3 (30.0) 6 (18.2) 0 (0.0)
 No Criteria Met 2 (20.0) 6 (18.2) 11 (78.6)
Current Medications, n (%)
 Antipsychotic 2 (20.0) 6 (18.2) 0 (0.0)
 Antidepressant 4 (40.0) 7 (21.2) 0 (0.0)
SIPS/SOPS, Mean (SD)
 Total Positive Symptoms 13.0 (3.2) 12.9 (3.8) 0.8 (1.2)
 Total Negative Symptoms 12.5 (6.3) 11.8 (6.1) 0.6 (0.9)
 GAF Current 50.0 (13.5) 58.8 (11.0) 86.9 (4.5)
Open in a new tab

CHR+rSI = Clinical high-risk for psychosis with recent suicidal ideation.

CHR−rSI = Clinical high-risk for psychosis without recent suicidal ideation.

SCID = Structured Clinical Interview for DSM-5 Disorders.

SIPS/SOPS = Structured Interview for Psychosis-Risk Syndromes/Scale of Psychosis-Risk Symptoms.

GAF = Global Assessment of Functioning.

*

p < 0.05 (CHR+rSI vs. CHR−rSI)

3.2. Severity of Suicidal Ideation

Of note, of the 10 CHR+rSI individuals, all endorsed the more passive “wish to be dead”; 2 reported having non-specific active suicidal thoughts; 2 endorsed specific suicidal ideation without intent; and 1 reported suicidal ideation with some intent but no plan (Fig. 1). While no analyses were done stratified by suicidal ideation severity, we highlight the prevalence of the “wish to be dead”, as it is consistent with our prior study of suicidal ideation in an earlier (and non-overlapping) CHR cohort also assessed with the C-SSRS (Gill et al., 2015), and may have ramifications for interpretation of results from language analyses.

Fig 1. Severity of recent suicidal ideation (SI) in clinical high-risk (CHR) individuals.

Fig 1.

Open in a new tab

CHR = clinical high risk for psychosis

SI = suicidal ideation

Bar Chart illustrating the distribution of severity of endorsed recent suicidal ideation. Among 43 total Clinical High Risk (CHR) participants, 33 reported having no recent suicidal ideation (past month); 10 reported having recent suicidal ideation (past month). Of the 10 individuals who had suicidal ideation in the past month, all 10 endorsed having at least a passive “Wish to be dead”. Two individuals reported having “Non-specific active suicidal thoughts” and “Suicidal ideation without the intent to act” as well. One participant endorsed having “Suicidal ideation with some intent but without plan” as well. These severity categories are measured directly in the Columbia Suicide Severity Rating Scale (C-SSRS).

3.3. Emotion Detection Analysis

Consistent with our hypothesis, CHR+rSI individuals (vs. CHR−rSI) used a greater frequency of words that had semantic similarity to “anger”, using both GoEmotions-trained BERT models (s = 0.5; p < 0.05) and zero-shot text classification (s = 0.5; p < 0.01) (Fig. 2a, 2c). By contrast, there was no difference between these CHR groups in using words with semantic similarity to “sadness”, again using both GoEmotions (s = 0.3; p = 0.40) and zero-shot text classification (s = 0.2; p = 0.82) (Fig. 2b, 2d). Further, using zero-shot text classification, no difference was found between the CHR groups for semantic similarity with “stress” (s = 0.1; p = 1.00) (Fig. 2e). Of note, using zero-shot text classification, “loneliness” was more indicative of language used by CHR+rSI individuals, though this effect did not remain after adjusting for sex (s = 0.3; p = 0.35) (Fig. 2f).

Fig 2. Emotion Detection Analyses Across Groups.

Fig 2.

Open in a new tab

HC = healthy controls

CHR = clinical high risk for psychosis

SI = suicidal ideation

*p < 0.05 (CHR with Recent SI vs. CHR without Recent SI)

Cumulative Distribution Function (CDF) for use of words with semantic similarity to hypothesized emotions in healthy controls (n=14, blue), Clinical High Risk (CHR) with recent suicidal ideation (n=10, red), and Clinical High Risk (CHR) without recent suicidal ideation (n=33, yellow). (A) “Anger” using GoEmotions, (B) “Sadness” using GoEmotions, (C) “Anger” using Zero Shot Text Classification, (D) “Sadness” using Zero Shot Text Classification, (E) “Stress” using Zero Shot Text Classification, (F) “Lonliness” using Zero Shot Text Classification. Use of words with semantic similarity to “anger” significantly differentiated the CHR with recent suicidal ideation from those without across both analytic approaches. CHR individuals with recent suicidal ideation consistently utilized more anger-related words (s = 0.5; p<0.05).

3.4. Part of Speech Analysis:

In respect to parts of speech, compared to CHR−rSI individuals, use of the word “I” in spoken language was actually decreased (s = 0.5; p = 0.06) (Fig. 3a) among CHR+rSI, contrary to our hypothesis that it would be more, based on the existing literature. Supplemental analyses were performed including the personal pronouns “me” and “mine”; however, individuals at CHR with recent suicidal ideation still utilized fewer first-person singular personal pronouns than those without recent suicidal ideation (s=0.5, p < 0.05) (Fig. S1).

Fig 3. Part of Speech Analyses Across Groups.

Fig 3.

Open in a new tab

HC = healthy controls

CHR = clinical high risk for psychosis

SI = suicidal ideation

**p < 0.01 (CHR with Recent SI vs. CHR without Recent SI)

Cumulative Distribution Function (CDF) for part of speech analyses in healthy controls (n=14, blue), Clinical High Risk (CHR) with recent suicidal ideation (n=10, red), and Clinical High Risk (CHR) without recent suicidal ideation (n=33, yellow). (A) Personal pronoun “I” use, (B) Modal use (e.g., “could”, “would”, “should”). Individuals at CHR with recent suicidal ideation utilized significantly fewer modals compared to the CHR without recent suicidal ideation group (s = 0.6, p < 0.01) and healthy controls (s = 0.6, p < 0.05).

In respect to exploratory analyses, only the use of modal words (e.g., “could”, “would”, “should”) were significantly different among groups, with CHR+rSI individuals using these words significantly less than both the CHR−rSI (s = 0.6, p < 0.01) and the healthy control groups (s = 0.6, p < 0.05) (Fig. 3b).

4. Discussion

The early stages of psychosis represent a critical period of suicide risk (Nielssen & Large, 2009; Nordentoft et al., 2004; Palmer et al., 2005; Upthegrove et al., 2010), including those at CHR (DeVylder et al., 2012; Gill et al., 2015; Hutton et al., 2011; Taylor et al., 2015). NLP serves as a promising and sensitive approach for the identification of linguistic markers of suicidality (Cohen et al., 2020; Corcoran & Cecchi, 2020; Homan et al., 2022; Pestian et al., 2016, 2020). Prior work has identified both semantic and syntactic language patterns associated with suicidal ideation, as described in a recent systematic review (Homan et al., 2022), including the increased use of words related to “anger”, “sadness”, “loneliness” and “stress” (Agurto et al., 2018; Coppersmith et al., 2015; Glenn et al., 2020; Ji et al., 2018; Nguyen et al., 2017; Pestian et al., 2016; Ramírez-Cifuentes et al., 2021; Shah et al., 2019) as well as greater first-person singular pronoun usage (Agurto et al., 2018; Coppersmith et al., 2015; Ji et al., 2018; Leavitt et al., 2021; Lekkas et al., 2021; O’Dea et al., 2021; Tadesse et al., 2020).

Our study is the first to use NLP to assess language patterns associated with suicidal ideation in individuals with CHR symptoms (Homan et al., 2022). It is relatively unique in that it also focused on transcripts of spoken language instead of written materials (social media posts, texts, etc.), and in using the C-SSRS (Homan et al., 2022). Nonetheless, consistent with prior studies in different cohorts using different methodologies (Agurto et al., 2018; Glenn et al., 2020; Homan et al., 2022; Ramírez-Cifuentes et al., 2021; Shah et al., 2019), CHR+rSI individuals, in open-ended qualitative interviews, had a significantly greater use of words that had semantic similarity to the word “anger”, as hypothesized, compared to both CHR−rSI and healthy individuals. Further, the association of anger with suicidal ideation was found using two different NLP approaches. The role of anger in suicide risk for young adults has also been reported with other psychiatric conditions (Daniel et al., 2009; Weniger et al., 2016). Our results are consistent with the view that anger may be a transdiagnostic suicide risk factor (Dillon et al., 2020).

This finding of the importance of anger is of great interest, as expressed anger is not an obvious characteristic of CHR teens and young adults. But pertaining to individuals with psychotic-like symptoms, the anger associated with suicidal ideation may be reflected by the “aggressive obsessions” that characterize a subset of individuals with CHR (DeVylder et al., 2012) or schizophrenia (Faragian et al., 2009). Our findings suggest that clinicians should be attentive to fluctuations in subthreshold anger-related sentiment when considering risk for suicidality. Future clinical applications for diminishing suicide risk in CHR individuals may include targeting sub-threshold anger as a means of early intervention, and incorporating principles of dialectical behavior therapy (DBT; Linehan et al., 1991), which target anger and SIB.

In contrast to “anger”, however, hypothesized associations of recent suicidal ideation with increased use of words with semantic similarity to “sadness”, “loneliness”, and “stress” were not found. For “sadness”, CHR individuals as a group used more words with semantic similarity to “sadness” than did healthy individuals – found as a trend using GoEmotions but clearly significant using zero-shot text classification. This suggests that sadness is something common to CHR individuals that does not differentiate those who endorse recent suicidal ideation from those who do not. For “stress”, studied with zero-shot text classification, we found that everyone – CHR+rSI, CHR−rSI, and healthy youths – had high usage of words with semantic similarity to “stress”. Finally, usage of words with semantic similarity to “loneliness” was greater among CHR+rSI (vs. CHR−rSI), but with an effect size of 0.3, which would require a much larger sample size to detect.

As for pronoun use, increased use of first-person singular pronouns such as “I” has been associated with suicidal ideation (Homan et al., 2022) in mostly written materials, including Reddit posts (Tadesse et al., 2020), Twitter (Ji et al., 2018; Vioules et al., 2018), forum posts (Nguyen et al., 2017), blogs (O’Dea et al., 2021), and suicide notes (Kim et al., 2019; Leavitt et al., 2021). Nonetheless, in this study, using transcripts of spoken language, we had the paradoxical finding that CHR+rSI individuals had significantly lower use of the first-person singular pronoun “I” compared to CHR−rSI, and comparable to healthy volunteers.

Possible reasons for our paradoxical finding on the use of “I” include differences in modes of elicitation of language, how suicidal ideation was measured, and the nature of the CHR cohort. It may be that for spoken language, the use of the word “I” does not reflect suicidal ideation, consistent with the one study based on clinical interviews (Pestian et al., 2020). Of note, Pestian and colleagues (2020) was also one of only two studies (of 75 in the Homan and colleagues (2022) review) that used the C-SSRS, such that how suicidal ideation is measured may also be relevant. Finally, CHR individuals may be different from other cohorts in the characteristics of their suicidal ideation and also in their use of the word “I”. The “wish to be dead” characterized all CHR+rSI, with only two persons having any plan or intent, similar to what we found in an earlier cohort (Gill et al., 2015). It may be that with more passive suicidal ideation, there is less of an emphasis on “I” and therefore less use of first-person singular pronouns in spoken language. This may also reflect disturbances in identity described using thematic analyses of many of these same transcripts (Saraç et al., 2021).

Finally, in our exploratory POS-Tag analyses, we found a significantly decreased use of modals (e.g., “could”, “would”, “should”) in spoken language among CHR+rSI compared to CHR−rSI individuals (p < 0.01) and healthy controls (p < 0.05). While these findings do not survive correction for multiple (n = 36) comparisons, they warrant further investigation in larger cohorts. The recent systematic review (Homan et al., 2022) identified only one study of “modals” and suicidal ideation (Dieltjens et al., 2014), a Dutch study of language use by patients seeking physician-assisted suicide that found increased use of modal verbs related to possibility, volition, and obligation (similar to “could”, “would”, and “should”). The opposite associations of modal use and suicidal ideation may reflect the differences between the passive “wish to be dead” among CHR individuals and the active seeking of relief from unbearable suffering among these Dutch patients seeking physician-assisted suicide. Overall, the decreased use of “I” and of modal words of “could”, “would” and “should” together is consistent with studies that show that among individuals in the online community, disruptions in the sense of continuity of the self into the future are associated with self-reports of suicidal ideation (Sokol and Eisenheim, 2016). Larger samples and finer grain identity assessments in CHR individuals are needed to test these hypotheses.

Overall, this study is the first study to use NLP to assess language patterns in spoken language of CHR individuals, who are well-characterized clinically, using the gold-standard C-SSRS. Further, state-of-the-art NLP approaches were used to assess semantic content, including BERT-based GoEmotions and zero-shot text classification, as well as POS-Tag methods, to both test hypothesized differences in first-person singular pronoun use and explore any other language patterns that might exist in the data. There are implications for therapeutics, both for CHR individuals and young people in distress more broadly, in that expressed anger in spoken language may warrant consideration of increased suicide risk, and therapies such as DBT that address both. Of note, the CHR+rSI and CHR−rSI individuals were similar in many ways, including age, race, ethnicity, antipsychotic medication use, and prodromal symptom severity, suggesting that NLP may be more sensitive to differences between these subgroups.

Nonetheless, several limitations exist, primarily a modest sample size, such that only large effect sizes could be detected in this first study of its kind of correlates of recent suicidal ideation in the CHR population. For example, the lack of significant difference in use of words with semantic similarity to “loneliness” among CHR+rSI individuals may be a result of Type 2 error. Childhood trauma was also not assessed, which has been associated with suicidal ideation in CHR cohorts (Taylor et al., 2015). Other covariates that warrant further study include female sex, depression diagnosis, and antidepressant use, which were all overrepresented among the CHR+rSI group. Further, some CHR−rSI individuals had a lifetime history of suicidal ideation and/or behavior; linguistic patterns may be different if the CHR−rSI group excluded these individuals. Finally, another limitation is that this study was done in an English-speaking cohort and may not be generalizable to other languages. For instance, in the recent systematic review, the use of “I” in written language was associated with suicidal ideation primarily only in English, but not in other languages, such as German (Instagram: Lekkas et al., 2021), Russian (texts: Litvinova et al., 2017) or Spanish (Facebook and Twitter: Sierra et al., 2022; though it was increased in a study of Korean suicide notes; Kim et al., 2019).

These limitations could be addressed in a large diverse prospective cohort study of CHR individuals who speak different languages. Also, beyond language structure and content, acoustic and prosodic features of spoken language can be assessed in respect to SIB. As an example, US veterans with suicidal thoughts (vs. those without) had speech that was flatter, less animated, more monotonous, duller, less energetic and breathier (Belouali et al., 2021). It would be useful to determine if similar features are associated with recent suicidal ideation in CHR individuals.

5. Conclusion

While preliminary in nature, this first study of linguistic correlates of suicidal ideation in a CHR sample, using NLP on naturally elicited speech yielded interesting findings in respect to anger. Language-based prediction models for suicide risk hold promise as they are inexpensive, and scalable, although ethical issues must be kept in mind. Building on this study, with extension into larger cohorts, and analyzing suicidal behavior, may aid in the early identification and prevention of suicide for individuals within and beyond the psychosis spectrum.

Supplementary Material

1

Acknowledgement

This work was funded by the National Institute of Mental Health (NIMH), titled Thought disorder and social cognition in clinical risk states for schizophrenia (R01MH107558-Suicide Supplement).

Role of the Funding Source

The funder had no role in study design, the collection, analysis, and interpretation of data, the writing of the report, or the decision to submit the article for publication.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of Interest

GC is employed by IBM Research, the research branch of IBM Corporation. MG serves as a consultant for Boehringer Ingleheim Pharmaceuticals. All other authors declare no competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

  1. Agurto C, Pataranutaporn P, Eyigoz EK, Stolovitzky G, & Cecchi G (2018, April 9). Predictive Linguistic Markers of Suicidality in Poets. IEEE International Conference on Semantic Computing. 10.1109/ICSC.2018.00051 [DOI] [Google Scholar]
  2. American Foundation for Suicide Prevention. (2019, December 25). Risk factors, protective factors, and warning signs. American Foundation for Suicide Prevention. https://afsp.org/risk-factors-protective-factors-and-warning-signs/ [Google Scholar]
  3. Badal VD, Graham SA, Depp CA, Shinkawa K, Yamada Y, Palinkas LA, Kim H-C, Jeste DV, & Lee EE (2020). Prediction of Loneliness in Older Adults Using Natural Language Processing: Exploring Sex Differences in Speech. The American Journal of Geriatric Psychiatry, 10.1016/j.jagp.2020.09.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bedi G, Carrillo F, Cecchi GA, Slezak DF, Sigman M, Mota NB, Ribeiro S, Javitt DC, Copelli M, & Corcoran CM (2015). Automated analysis of free speech predicts psychosis onset in high-risk youths. Npj Schizophrenia, 1(1), 1–7. 10.1038/npjschz.2015.30 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Belouali A, Gupta S, Sourirajan V, Yu J, Allen N, Alaoui A, Dutton MA, & Reinhard MJ (2021). Acoustic and language analysis of speech for suicidal ideation among US veterans. BioData Mining, 14(1), 11. 10.1186/s13040-021-00245-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Ben-David S, Birnbaum ML, Eilenberg ME, DeVylder JE, Gill KE, Schienle J, Azimov N, Lukens EP, Davidson L, & Corcoran CM (2014). The Subjective Experience of Youths at Clinically High Risk of Psychosis: A Qualitative Study. Psychiatric Services, 65(12), 1499–1501. 10.1176/appi.ps.201300527 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Burkhardt H, Pullmann M, Hull T, Aren P, & Cohen T (2022). Comparing emotion feature extraction approaches for predicting depression and anxiety. Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology, 105–115. 10.18653/v1/2022.clpsych-E9 [DOI] [Google Scholar]
  8. Chang WC, Chen ESM, Hui CLM, Chan SKW, Lee EHM, & Chen EYH (2014). The relationships of suicidal ideation with symptoms, neurocognitive function, and psychological factors in patients with first-episode psychosis. Schizophrenia Research, 157(1), 12–18. 10.1016/j.schres.2014.06.009 [DOI] [PubMed] [Google Scholar]
  9. Centers for Disease Control and Prevention. Web-based Injury Statistics Query and Reporting System (WISQARS) Fatal Injury Reports. (2020, February 20). Retrieved February 9, 2021, from https://webappa.cdc.gov/sasweb/ncipc/mortrate.html
  10. Cohen J, Wright-Berryman J, Rohlfs L, Wright D, Campbell M, Gingrich D, Santel D, & Pestian J (2020). A Feasibility Study Using a Machine Learning Suicide Risk Prediction Model Based on Open-Ended Interview Language in Adolescent Therapy Sessions. International Journal of Environmental Research and Public Health, 17(21), Article 21. 10.3390/ijerph17218187 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Coppersmith G, Leary R, Whyne E, & Wood T (2015). Quantifying suicidal ideation via language usage on social media. Joint Statistics Meetings Proceedings, Statistical Computing Section, JSM, 110. [Google Scholar]
  12. Corcoran CM, Carrillo F, Fernández-Slezak D, Bedi G, Klim C, Javitt DC, Bearden CE, & Cecchi GA (2018). Prediction of psychosis across protocols and risk cohorts using automated language analysis. World Psychiatry, 17(1), 67–75. 10.1002/wps.20491 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Corcoran CM, & Cecchi GA (2020). Using Language Processing and Speech Analysis for the Identification of Psychosis and Other Disorders. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 5(8), 770–779. 10.1016/j.bpsc.2020.06.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Correll CU, Hauser M, Auther AM, & Cornblatt BA (2010). Research in people with psychosis risk syndrome: A review of the current evidence and future directions. Journal of Child Psychology and Psychiatry, 51(4), 390–431. 10.1111/j.1469-7610.2010.02235.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Daniel SS, Goldston DB, Erkanli A, Franklin JC, & Mayfield AM (2009). Trait Anger, Anger Expression, and Suicide Attempts Among Adolescents and Young Adults: A Prospective Study. Journal of Clinical Child & Adolescent Psychology, 38(5), 661–671. 10.1080/15374410903103494 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Demszky D, Movshovitz-Attias D, Ko J, Cowen A, Nemade G, & Ravi S (2020). GoEmotions: A Dataset of Fine-Grained Emotions (arXiv:2005.00547). arXiv. http://arxiv.org/abs/2005.00547 [Google Scholar]
  17. Devlin J, Chang M-W, Lee K, & Toutanova K (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. ArXiv Preprint ArXiv:1810.04805. [Google Scholar]
  18. DeVylder JE, Oh AJ, Ben-David S, Azimov N, Harkavy-Friedman JM, & Corcoran CM (2012). Obsessive compulsive symptoms in individuals at clinical risk for psychosis: Association with depressive symptoms and suicidal ideation. Schizophrenia Research, 140(1), 110–113. 10.1016/j.schres.2012.07.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Dieltjens SM, Heynderickx PC, Dees MK, & Vissers KC (2014). Linguistic Analysis of Face-to-Face Interviews with Patients with An Explicit Request for Euthanasia, their Closest Relatives, and their Attending Physicians: The Use of Modal Verbs in Dutch. Pain Practice, 14(4), 324–331. 10.1111/papr.12076 [DOI] [PubMed] [Google Scholar]
  20. Dillon KH, Van Voorhees EE, & Elbogen EB (2020). Associations between anger and suicidal ideation and attempts: A prospective study using the National Epidemiologic Survey on Alcohol and Related Conditions. Journal of Psychiatric Research, 122, 17–21. 10.1016/j.jpsychires.2019.12.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Faragian S, Pashinian A, Fuchs C, & Poyurovsky M (2009). Obsessive-compulsive symptom dimensions in schizophrenia patients with comorbid obsessive–compulsive disorder. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 33(6), 1009–1012. 10.1016/j.pnpbp.2009.05.008 [DOI] [PubMed] [Google Scholar]
  22. First MB, Williams JB, Karg RS, & Spitzer RL (2015). Structured clinical interview for DSM-5—Research version (SCID-5 for DSM-5, research version; SCID-5-RV). Arlington, VA: American Psychiatric Association, 1–94. [Google Scholar]
  23. Forman EM, Berk MS, Henriques GR, Brown GK, & Beck AT (2004). History of multiple suicide attempts as a behavioral marker of severe psychopathology. American Journal of Psychiatry, 161(3), 437–443. [DOI] [PubMed] [Google Scholar]
  24. Gill KE, Quintero JM, Poe SL, Moreira AD, Brucato G, Corcoran CM, & Girgis RR (2015). Assessing suicidal ideation in individuals at clinical high risk for psychosis. Schizophrenia Research, 165(2), 152–156. 10.1016/j.schres.2015.04.022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Glenn JJ, Nobles AL, Barnes LE, & Teachman BA (2020). Can Text Messages Identify Suicide Risk in Real Time? A Within-Subjects Pilot Examination of Temporally Sensitive Markers of Suicide Risk. Clinical Psychological Science, 8(4), 704–722. 10.1177/2167702620906146 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Hedegaard H, Curtin SC, & Warner M (2021). Suicide Mortality in the United States, 1999-2019. NCHS Data Brief 398, 1–8. [PubMed] [Google Scholar]
  27. Homan S, Gabi M, Klee N, Bachmann S, Moser A-M, Michel S, Bertram A-M, Maatz A, Seiler G, & Stark E (2022). Linguistic features of suicidal thoughts and behaviors: A systematic review. Clinical Psychology Review, 102161. [DOI] [PubMed] [Google Scholar]
  28. Hutton P, Bowe S, Parker S, & Ford S (2011). Prevalence of suicide risk factors in people at ultra-high risk of developing psychosis: A service audit. Early Intervention in Psychiatry, 5(4), 375–380. 10.1111/j.1751-7893.2011.00302.x [DOI] [PubMed] [Google Scholar]
  29. Ji S, Yu CP, Fung S, Pan S, & Long G (2018). Supervised Learning for Suicidal Ideation Detection in Online User Content. Complexity, 2018, e6157249. 10.1155/2018/6157249 [DOI] [Google Scholar]
  30. Kim K, Choi S, Lee J, & Sea J (2019). Differences in linguistic and psychological characteristics between suicide notes and diaries. The Journal of General Psychology, 146(4), 391–416. 10.1080/00221309.2019.1590304 [DOI] [PubMed] [Google Scholar]
  31. Leavitt J, Hong JH, & Walker RL (2021). Paradoxical positivity: Suicide notes use less distressed language than blogs about depression, suicidal thoughts, and even cooking. Suicide and Life-Threatening Behavior, 51(5), 1005–1014. 10.1111/sltb.12786 [DOI] [PubMed] [Google Scholar]
  32. Lekkas D, Klein RJ, & Jacobson NC (2021). Predicting acute suicidal ideation on Instagram using ensemble machine learning models. Internet Interventions, 25, 100424. 10.1016/j.invent.2021.100424 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Liang M, Zhou J, Sun Y, & He L (2021). Working with Few Samples: Methods that Help Analyze Social Attitude and Personal Emotion. 2021 IEEE 24th International Conference on Computer Supported Cooperative Work in Design (CSCWD), 1135–1140. 10.1109/CSCWD49262.2021.9437799 [DOI] [Google Scholar]
  34. Linehan MM, Armstrong HE, Suarez A, Allmon D, & Heard HL (1991). Cognitive-Behavioral Treatment of Chronically Parasuicidal Borderline Patients. Archives of General Psychiatry, 48(12), 1060–1064. 10.1001/archpsyc.1991.01810360024003 [DOI] [PubMed] [Google Scholar]
  35. Litvinova TA, Seredin PV, Litvinova OA, Romanchenko OV, Litvinova TA, Seredin PV, Litvinova OA, & Romanchenko OV (2017). Identification of Suicidal Tendencies of Individuals Based on the Quantitative Analysis of their Internet Texts. Computación y Sistemas, 21(2), 243–252. 10.13053/cys-21-2-2721 [DOI] [Google Scholar]
  36. Miller TJ, McGlashan TH, Rosen JL, Cadenhead K, Ventura J, McFarlane W, Perkins DO, Pearlson GD, & Woods SW (2003). Prodromal Assessment With the Structured Interview for Prodromal Syndromes and the Scale of Prodromal Symptoms: Predictive Validity, Interrater Reliability, and Training to Reliability. Schizophrenia Bulletin, 29(4), 703–715. 10.1093/oxfordjournals.schbul.a007040 [DOI] [PubMed] [Google Scholar]
  37. Nguyen T, O’Dea B, Larsen M, Phung D, Venkatesh S, & Christensen H (2017). Using linguistic and topic analysis to classify sub-groups of online depression communities. Multimedia Tools and Applications, 76(8), 10653–10676. 10.1007/s11042-015-3128-x [DOI] [Google Scholar]
  38. Nielssen OB, & Large MM (2009). Untreated psychotic illness in the survivors of violent suicide attempts. Early Intervention in Psychiatry, 3(2), 116–122. 10.1111/j.1751-7893.2009.00117.x [DOI] [PubMed] [Google Scholar]
  39. Nordentoft M, Laursen TM, Agerbo E, Qin P, Høyer EH, & Mortensen PB (2004). Change in suicide rates for patients with schizophrenia in Denmark, 1981-97: Nested case-control study. BMJ, 329(7460), 261. 10.1136/bmj.38133.622488.63 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. O’Dea B, Boonstra TW, Larsen ME, Nguyen T, Venkatesh S, & Christensen H (2021). The relationship between linguistic expression in blog content and symptoms of depression, anxiety, and suicidal thoughts: A longitudinal study. PLOS ONE, 16(5), e0251787. 10.1371/journal.pone.0251787 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Palmer BA, Pankratz VS, & Bostwick JM (2005). The Lifetime Risk of Suicide in Schizophrenia: A Reexamination. Archives of General Psychiatry, 62(3), 247–253. 10.1001/archpsyc.62.3.247 [DOI] [PubMed] [Google Scholar]
  42. Park CHK, Lee JW, Lee SY, Moon J, Jeon D-W, Shim S-H, Cho S-J, Kim SG, Lee J, Paik J-W, Kim M-H, You S, Jeon HJ, Rhee SJ, Kim MJ, Kim J, & Ahn YM (2020). Suicide risk factors across suicidal ideators, single suicide attempters, and multiple suicide attempters. Journal of Psychiatric Research, 131, 1–8. 10.1016/j.jpsychires.2020.08.018 [DOI] [PubMed] [Google Scholar]
  43. Pestian J, Grupp-Phelan J, Bretonnel Cohen K, Meyers G, Richey LA, Matykiewicz P, & Sorter MT (2016). A Controlled Trial Using Natural Language Processing to Examine the Language of Suicidal Adolescents in the Emergency Department. Suicide and Life-Threatening Behavior, 46(2), 154–159. 10.1111/sltb.12180 [DOI] [PubMed] [Google Scholar]
  44. Pestian J, Santel D, Sorter M, Bayram U, Connolly B, Glauser T, DelBello M, Tamang S, & Cohen K (2020). A Machine Learning Approach to Identifying Changes in Suicidal Language. Suicide and Life-Threatening Behavior, 50(5), 939–947. 10.1111/sltb.12642 [DOI] [PubMed] [Google Scholar]
  45. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, Currier GW, Melvin GA, Greenhill L, Shen S, & Mann JJ (2011). The Columbia–Suicide Severity Rating Scale: Initial Validity and Internal Consistency Findings From Three Multisite Studies With Adolescents and Adults. American Journal of Psychiatry, 168(12), 1266–1277. 10.1176/appi.ajp.2011.10111704 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Ramírez-Cifuentes D, Largeron C, Tissier J, Baeza-Yates R, & Freire A (2021). Enhanced Word Embedding Variations for the Detection of Substance Abuse and Mental Health Issues on Social Media Writings. IEEE Access, 9, 130449–130471. 10.1109/ACCESS.2021.3112102 [DOI] [Google Scholar]
  47. Santorini B (1990). Part-of-Speech Tagging Guidelines for the Penn Treebank Project (3rd Revision). Technical Reports (CIS). [Google Scholar]
  48. Saraç C, DeLuca J, Bilgrami Z, Herrera S, Myers J, Dobbs M, Haas S, Todd T, Srivastava A, Jespersen R, Shaik R, Landa Y, Davidson L, Pavlo A, & Corcoran C (2021). A qualitative study on identity in individuals at clinical high-risk for psychosis: “… Why does it have to be one thing?” In Psychiatric Rehabilitation Journal. 10.1037/prj0000492 [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Shah AP, Vaibhav V, Sharma V, Al Ismail M, Girard J, & Morency L-P (2019). Multimodal Behavioral Markers Exploring Suicidal Intent in Social Media Videos. 2019 International Conference on Multimodal Interaction, 409–413. 10.1145/3340555.3353718 [DOI] [Google Scholar]
  50. Sierra G, Andrade-Palos P, Bel-Enguix G, Osornio-Arteaga A, Cabrera-Mora A, García-Nieto L, & Sierra-Aparicio T (2022). Suicide Risk Factors: A Language Analysis Approach in Social Media. Journal of Language and Social Psychology, 41(3), 312–330. 10.1177/0261927X211036171 [DOI] [Google Scholar]
  51. Sokol Y, & Eisenheim E (2016). The Relationship Between Continuous Identity Disturbances, Negative Mood, and Suicidal Ideation. The Primary Care Companion for CNS Disorders, 18(1), 26150. 10.4088/PCC.15m01824 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Tadesse MM, Lin H, Xu B, & Yang L (2020). Detection of Suicide Ideation in Social Media Forums Using Deep Learning. Algorithms, 13(1), Article 1. 10.3390/a13010007 [DOI] [Google Scholar]
  53. Tausczik YR, & Pennebaker JW (2010). The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods. Journal of Language and Social Psychology, 29(1), 24–54. 10.1177/0261927X09351676 [DOI] [Google Scholar]
  54. Taylor PJ, Hutton P, & Wood L (2015). Are people at risk of psychosis also at risk of suicide and self-harm? A systematic review and meta-analysis. Psychological Medicine, 45(5), 911–926. 10.1017/S0033291714002074 [DOI] [PubMed] [Google Scholar]
  55. Too LS, Spittal MJ, Bugeja L, Reifels L, Butterworth P, & Pirkis J (2019). The association between mental disorders and suicide: A systematic review and meta-analysis of record linkage studies. Journal of Affective Disorders, 259, 302–313. 10.1016/j.jad.2019.08.054 [DOI] [PubMed] [Google Scholar]
  56. Tuarob S, Noraset T, & Tawichsri T (2022). Using Large-Scale Social Media Data for Popidation-Level Mental Health Monitoring and Public Sentiment Assessment: A Case Study of Thailand. Puey Ungphakorn Institute for Economic Research. [Google Scholar]
  57. Turcan E, Muresan S, & McKeown K (2021). Emotion-Infused Models for Explainable Psychological Stress Detection. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2895–2909. 10.18653/v1/2021.naacl-main.230 [DOI] [Google Scholar]
  58. Upthegrove R, Birchwood M, Ross K, Brunett K, McCollum R, & Jones L (2010). The evolution of depression and suicidality in first episode psychosis. Acta Psychiatrica Scandinavica, 122(3), 211–218. 10.1111/j.1600-0447.2009.01506.x [DOI] [PubMed] [Google Scholar]
  59. Van Heeringen K, Hawton K, & Williams JMG (2000). Pathways to suicide: An integrative approach. The International Handbook of Suicide and Attempted Suicide, 223–234. [Google Scholar]
  60. Vioules MJ, Moulahi B, Aze J, & Bringay S (2018). Detection of suicide-related posts in Twitter data streams. IBM Journal of Research and Development, 62(1), 7:1–7:12. 10.1147/JRD.2017.2768678 [DOI] [Google Scholar]
  61. Wawer A, Chojnicka I, Okruszek L, & Sarzynska-Wawer J (2022). Single and Cross-Disorder Detection for Autism and Schizophrenia. Cognitive Computation, 14(1), 461–473. 10.1007/s12559-021-09834-9 [DOI] [Google Scholar]
  62. Weniger J, Distelberg B, & Vaswani1 D (2016). Depression And Anger As Risk Factors For Suicide With Inpatient Adolescents. Suicidology Online, 7, 1–10. [Google Scholar]
  63. Yates K, Lång U, Cederlöf M, Boland F, Taylor P, Cannon M, McNicholas F, DeVylder J, & Kelleher I (2019). Association of Psychotic Experiences With Subsequent Risk of Suicidal Ideation, Suicide Attempts, and Suicide Deaths: A Systematic Review and Meta-analysis of Longitudinal Population Studies. JAMA Psychiatry, 76(2), 180–189. 10.1001/jamapsychiatry.2018.3514 [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Yin W, Hay J, & Roth D (2019). Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach (arXiv:1909.00161). arXiv. 10.48550/arXiv.1909.00161 [DOI] [Google Scholar]
  65. Zaheer J, Olfson M, Mallia E, Lam JSH, de Oliveira C, Rudoler D, Carvalho AF, Jacob BJ, Juda A, & Kurdyak P (2020). Predictors of suicide at time of diagnosis in schizophrenia spectrum disorder: A 20-year total population study in Ontario, Canada. Schizophrenia Research, 222, 382–388. 10.1016/j.schres.2020.04.025 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1883679-supplement-1.docx (50KB, docx)

RESOURCES